2024-08-16T18:50:15.983 INFO:root:teuthology version: 0.0.1.dev427+g4e29fcc 2024-08-16T18:50:16.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-16T18:50:16.093 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7855996' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: d863d21c9d1db9c3453c09dacd68766969720a44 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d863d21c9d1db9c3453c09dacd68766969720a44 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 998c2f980771f4723d937d5cd7891f92dd116699 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7387 sha1: d863d21c9d1db9c3453c09dacd68766969720a44 sleep_before_teardown: 0 subset: 143/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 998c2f980771f4723d937d5cd7891f92dd116699 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaYCJnFueS9H9R0e3UbTfqz8jl4azIvgWRD2AI1OA79L4aFtgBe0WUDQT1TGo1Y+3Rkms/zDC0DEFrc/Wz++WY= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYZ8iekaa4MlmurW1QTo0bvg4wByC5EQvS7UOg9vjGI+H/CM9WnMCpEYMYW86MtVV5cdo2AhGlRds7G5NqcA+k= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaXrEStRgye+/14/cNv9PtlJIBTg2BJgDYof4kU0sgkiIBa4VSECcKjTLMmt/ChABuzIiontQ2feK7vNw5AFE0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 4e29fccbc48636195c0aac083d870323c5832411 timestamp: 2024-08-14_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-16T18:50:16.094 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa; will attempt to use it 2024-08-16T18:50:16.094 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks 2024-08-16T18:50:16.094 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-16T18:50:16.096 INFO:teuthology.task.internal:Checking packages... 2024-08-16T18:50:16.116 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd863d21c9d1db9c3453c09dacd68766969720a44' 2024-08-16T18:50:16.116 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-16T18:50:16.116 INFO:teuthology.packaging:ref: None 2024-08-16T18:50:16.116 INFO:teuthology.packaging:tag: None 2024-08-16T18:50:16.116 INFO:teuthology.packaging:branch: squid 2024-08-16T18:50:16.117 INFO:teuthology.packaging:sha1: d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:50:16.117 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-08-16T18:50:16.313 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1378.g567d40c0 2024-08-16T18:50:16.314 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-16T18:50:16.321 INFO:teuthology.task.internal:no buildpackages task found 2024-08-16T18:50:16.321 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-16T18:50:16.339 INFO:teuthology.task.internal:Saving configuration 2024-08-16T18:50:16.358 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-16T18:50:16.365 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-16T18:50:16.390 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-16 18:45:06.817813', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBaYCJnFueS9H9R0e3UbTfqz8jl4azIvgWRD2AI1OA79L4aFtgBe0WUDQT1TGo1Y+3Rkms/zDC0DEFrc/Wz++WY='} 2024-08-16T18:50:16.415 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-16 18:45:06.815801', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYZ8iekaa4MlmurW1QTo0bvg4wByC5EQvS7UOg9vjGI+H/CM9WnMCpEYMYW86MtVV5cdo2AhGlRds7G5NqcA+k='} 2024-08-16T18:50:16.449 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-16 18:45:06.819352', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaXrEStRgye+/14/cNv9PtlJIBTg2BJgDYof4kU0sgkiIBa4VSECcKjTLMmt/ChABuzIiontQ2feK7vNw5AFE0='} 2024-08-16T18:50:16.450 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-16T18:50:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-16T18:50:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-16T18:50:16.456 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-16T18:50:16.456 INFO:teuthology.run_tasks:Running task console_log... 2024-08-16T18:50:16.537 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f043d5efbe0>, signals=[15]) 2024-08-16T18:50:16.537 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-16T18:50:16.543 INFO:teuthology.task.internal:Opening connections... 2024-08-16T18:50:16.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T18:50:16.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:50:16.621 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-08-16T18:50:16.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:50:16.698 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-08-16T18:50:16.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:50:16.771 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-16T18:50:16.778 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-08-16T18:50:16.796 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-08-16T18:50:16.797 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-08-16T18:50:16.855 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-16T18:50:16.856 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-16T18:50:16.857 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-08-16T18:50:16.887 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-08-16T18:50:16.905 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-08-16T18:50:16.905 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-16T18:50:16.963 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-16T18:50:16.964 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-16T18:50:16.964 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-08-16T18:50:16.987 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-08-16T18:50:17.005 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-08-16T18:50:17.006 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-08-16T18:50:17.062 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-16T18:50:17.063 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-16T18:50:17.064 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-08-16T18:50:17.087 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-16T18:50:17.098 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-16T18:50:17.103 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-16T18:50:17.103 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-08-16T18:50:17.106 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-08-16T18:50:17.108 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-08-16T18:50:17.125 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-16T18:50:17.131 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-16T18:50:17.131 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-08-16T18:50:17.163 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-08-16T18:50:17.166 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-08-16T18:50:17.330 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-16T18:50:17.408 INFO:teuthology.run_tasks:Running task kernel... 2024-08-16T18:50:17.421 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-16T18:50:17.421 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-08-16T18:50:17.421 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-08-16T18:50:17.422 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-16T18:50:17.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:50:17.439 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-08-16T18:50:17.499 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-437.el9.x86_64 2024-08-16T18:50:17.499 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-437.el9.x86_64 2024-08-16T18:50:17.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-08-16T18:50:19.851 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-08-16T18:50:21.869 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-08-16T18:50:27.053 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.6 MB 00:01 2024-08-16T18:50:28.762 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 18 kB 00:00 2024-08-16T18:50:29.991 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 20 MB/s | 22 MB 00:01 2024-08-16T18:50:35.730 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-08-16T18:50:37.770 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-16T18:50:37.770 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-08-16T18:50:37.867 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-08-16T18:50:37.884 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout:Install 4 Packages 2024-08-16T18:50:37.885 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:50:37.886 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 84 M 2024-08-16T18:50:37.886 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 126 M 2024-08-16T18:50:37.886 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-08-16T18:50:38.398 INFO:teuthology.orchestra.run.smithi062.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 4.0 MB/s | 1.5 MB 00:00 2024-08-16T18:50:39.182 INFO:teuthology.orchestra.run.smithi062.stdout:(2/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 31 MB/s | 36 MB 00:01 2024-08-16T18:50:39.449 INFO:teuthology.orchestra.run.smithi062.stdout:(3/4): kernel-core-5.14.0-480.el9.x86_64.rpm 12 MB/s | 17 MB 00:01 2024-08-16T18:50:40.099 INFO:teuthology.orchestra.run.smithi062.stdout:(4/4): kernel-modules-core-5.14.0-480.el9.x86_6 18 MB/s | 30 MB 00:01 2024-08-16T18:50:40.099 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-08-16T18:50:40.099 INFO:teuthology.orchestra.run.smithi062.stdout:Total 38 MB/s | 84 MB 00:02 2024-08-16T18:50:40.866 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-08-16T18:50:41.292 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-08-16T18:50:41.292 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-08-16T18:50:42.331 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-08-16T18:50:42.332 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-08-16T18:50:44.944 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-08-16T18:50:45.642 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-08-16T18:50:45.740 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-16T18:50:48.545 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-16T18:50:48.690 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-16T18:50:53.902 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-16T18:50:53.966 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:50:59.193 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:51:30.485 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:51:30.527 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:51:30.938 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:51:30.938 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-08-16T18:51:30.938 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-16T18:51:30.938 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-16T18:51:31.760 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-08-16T18:51:31.760 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:51:31.760 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-08-16T18:51:31.760 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-16T18:51:31.761 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-08-16T18:51:31.761 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-08-16T18:51:31.761 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-08-16T18:51:31.761 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:51:31.761 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-08-16T18:51:32.122 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-08-16T18:51:32.205 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-16T18:51:32.206 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-16T18:51:32.206 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-08-16T18:51:32.206 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-16T18:51:32.206 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-16T18:51:32.207 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-08-16T18:51:32.207 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2024-08-16T18:51:32.207 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-08-16T18:51:32.265 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-16T18:51:32.266 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-16T18:51:32.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-08-16T18:51:32.308 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-16T18:51:32.309 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-08-16T18:51:32.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:51:32.310 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-08-16T18:51:32.310 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-16T18:51:32.310 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 5.14.0-480.el9.x86_64 2024-08-16T18:51:32.311 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-16T18:51:32.910 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-08-16T18:51:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-16T18:51:34.342 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-08-16T18:51:34.344 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-08-16T18:51:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.jK3N3eJJro 2024-08-16T18:51:34.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jK3N3eJJro 2024-08-16T18:51:34.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.jK3N3eJJro 2024-08-16T18:51:34.567 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.jK3N3eJJro is 6KB 2024-08-16T18:51:34.615 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.jK3N3eJJro 2024-08-16T18:51:34.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-08-16T18:51:34.693 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-08-16T18:51:34.693 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-08-16T18:51:34.693 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-08-16T18:51:34.693 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-08-16T18:51:34.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-08-16T18:51:34.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-08-16T18:52:04.849 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-16T18:52:04.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:52:04.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:52:39.519 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-08-16T18:52:48.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:52:48.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:53:06.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-08-16T18:53:18.915 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:53:18.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:53:19.315 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-08-16T18:53:19.901 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:53:19.901 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-08-16T18:53:19.902 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-08-16T18:53:19.958 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-480.el9.x86_64 2024-08-16T18:53:19.958 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-08-16T18:53:19.959 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-16T18:53:19.959 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-16T18:53:20.960 DEBUG:teuthology.parallel:result is None 2024-08-16T18:53:20.960 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-16T18:53:20.967 INFO:teuthology.task.internal:Creating test directory... 2024-08-16T18:53:20.967 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-16T18:53:20.970 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-16T18:53:20.973 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-16T18:53:20.994 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-16T18:53:21.001 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-16T18:53:21.010 INFO:teuthology.task.internal:Creating archive directory... 2024-08-16T18:53:21.010 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-16T18:53:21.026 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-16T18:53:21.030 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-16T18:53:21.159 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-16T18:53:21.166 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-16T18:53:21.166 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-16T18:53:21.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:53:21.183 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-16T18:53:21.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:53:21.203 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-16T18:53:21.205 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-16T18:53:21.226 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-16T18:53:21.279 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.282 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.294 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.301 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.309 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.310 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-16T18:53:21.312 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-16T18:53:21.319 INFO:teuthology.task.internal:Configuring sudo... 2024-08-16T18:53:21.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-16T18:53:21.321 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-16T18:53:21.324 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-16T18:53:21.369 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-16T18:53:21.378 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-16T18:53:21.378 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-16T18:53:21.389 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-16T18:53:21.391 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-16T18:53:21.425 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:21.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:21.530 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:21.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:21.655 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T18:53:21.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-16T18:53:21.752 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:21.792 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:21.842 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:21.926 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:21.967 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T18:53:21.967 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-16T18:53:22.032 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:22.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-16T18:53:22.129 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:22.212 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-16T18:53:22.253 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T18:53:22.254 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-16T18:53:22.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-08-16T18:53:22.321 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-08-16T18:53:22.323 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-08-16T18:53:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T18:53:22.370 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T18:53:22.394 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T18:53:22.865 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-16T18:53:22.872 INFO:teuthology.task.internal:Starting timer... 2024-08-16T18:53:22.872 INFO:teuthology.run_tasks:Running task pcp... 2024-08-16T18:53:22.883 INFO:teuthology.run_tasks:Running task selinux... 2024-08-16T18:53:22.891 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-16T18:53:22.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-08-16T18:53:22.945 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-08-16T18:53:22.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-08-16T18:53:23.010 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-08-16T18:53:23.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-08-16T18:53:23.073 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-08-16T18:53:23.075 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-16T18:53:23.075 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-08-16T18:53:23.096 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-08-16T18:53:23.097 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-08-16T18:53:23.119 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-08-16T18:53:23.119 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-08-16T18:53:23.147 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-08-16T18:53:23.147 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-08-16T18:53:23.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T18:53:23.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:53:23.175 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T18:53:23.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:53:23.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T18:53:23.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T18:53:23.230 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-16T18:53:23.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-08-16T18:53:23.308 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-08-16T18:53:23.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-08-16T18:53:23.365 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-16T18:53:23.375 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-16T18:53:23.377 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-16T18:53:23.489 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-16T18:53:23.500 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-16T18:53:23.501 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi155.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-16T18:58:20.111 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-08-16T18:58:20.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:58:20.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:58:20.197 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-08-16T18:58:20.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-08-16T18:58:20.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-08-16T18:58:20.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:58:20.355 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-08-16T18:58:20.438 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-08-16T18:58:20.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-16T18:58:20.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-16T18:58:20.518 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-08-16T18:58:20.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-16T18:58:20.608 INFO:teuthology.run_tasks:Running task clock... 2024-08-16T18:58:20.618 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-16T18:58:20.619 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-16T18:58:20.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T18:58:20.622 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-16T18:58:20.622 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T18:58:20.624 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-16T18:58:20.624 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T18:58:20.653 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-16T18:58:20.654 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-16T18:58:20.667 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-16T18:58:20.670 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-16T18:58:20.677 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-16T18:58:20.691 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-08-16T18:58:20.691 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-16T18:58:20.697 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-08-16T18:58:20.701 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-08-16T18:58:20.710 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-08-16T18:58:20.715 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-08-16T18:58:20.715 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-16T18:58:20.725 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-16T18:58:20.725 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-08-16T18:58:20.729 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-16T18:58:20.739 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-16T18:58:20.740 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-16T18:58:20.752 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-16T18:58:20.781 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.785 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.799 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-08-16T18:58:20.802 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T18:58:20.803 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-08-16T18:58:20.803 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.803 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.803 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.803 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.806 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-08-16T18:58:20.809 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T18:58:20.809 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-08-16T18:58:20.809 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.809 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.810 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.810 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T18:58:20.810 INFO:teuthology.run_tasks:Running task install... 2024-08-16T18:58:20.817 DEBUG:teuthology.task.install:project ceph 2024-08-16T18:58:20.818 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44'}} 2024-08-16T18:58:20.818 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44'} 2024-08-16T18:58:20.818 INFO:teuthology.task.install:Using flavor: default 2024-08-16T18:58:20.823 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-16T18:58:20.823 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-16T18:58:20.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44', 'tag': None, 'wait_for_package': False} 2024-08-16T18:58:20.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:58:20.826 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44', 'tag': None, 'wait_for_package': False} 2024-08-16T18:58:20.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:58:20.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44', 'tag': None, 'wait_for_package': False} 2024-08-16T18:58:20.827 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:58:20.972 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/ 2024-08-16T18:58:20.972 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1363.gd863d21c 2024-08-16T18:58:20.975 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/ 2024-08-16T18:58:20.976 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1363.gd863d21c 2024-08-16T18:58:20.988 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/ 2024-08-16T18:58:20.989 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1363.gd863d21c 2024-08-16T18:58:21.084 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-16T18:58:21.084 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T18:58:21.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-16T18:58:21.089 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-16T18:58:21.089 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T18:58:21.089 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-16T18:58:21.096 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-16T18:58:21.096 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T18:58:21.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-16T18:58:21.112 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-16T18:58:21.112 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d863d21c9d1db9c3453c09dacd68766969720a44/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-16T18:58:21.113 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-16T18:58:21.114 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d863d21c9d1db9c3453c09dacd68766969720a44/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-16T18:58:21.121 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-16T18:58:21.122 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d863d21c9d1db9c3453c09dacd68766969720a44/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-16T18:58:21.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-16T18:58:21.180 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-16T18:58:21.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-16T18:58:21.255 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-16T18:58:21.256 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-16T18:58:21.258 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-16T18:58:21.318 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-08-16T18:58:21.319 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-08-16T18:58:21.321 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-08-16T18:58:21.321 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-08-16T18:58:21.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-08-16T18:58:21.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-08-16T18:58:21.544 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-08-16T18:58:21.546 INFO:teuthology.orchestra.run.smithi155.stdout:45 files removed 2024-08-16T18:58:21.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-16T18:58:21.573 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-16T18:58:21.580 INFO:teuthology.orchestra.run.smithi062.stdout:45 files removed 2024-08-16T18:58:21.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-16T18:58:22.155 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 260 kB/s | 84 kB 00:00 2024-08-16T18:58:22.160 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 260 kB/s | 84 kB 00:00 2024-08-16T18:58:22.218 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 263 kB/s | 84 kB 00:00 2024-08-16T18:58:22.463 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-08-16T18:58:22.494 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-08-16T18:58:22.569 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 39 kB/s | 12 kB 00:00 2024-08-16T18:58:22.789 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-08-16T18:58:22.819 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-08-16T18:58:22.894 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-08-16T18:58:23.444 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-08-16T18:58:23.578 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-08-16T18:58:23.947 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.2 MB 00:01 2024-08-16T18:58:25.649 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-08-16T18:58:25.688 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-08-16T18:58:26.812 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 20 MB 00:02 2024-08-16T18:58:29.950 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.6 MB 00:00 2024-08-16T18:58:29.963 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.6 MB 00:00 2024-08-16T18:58:31.204 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.6 MB 00:00 2024-08-16T18:58:31.417 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 18 kB 00:00 2024-08-16T18:58:31.939 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 18 kB 00:00 2024-08-16T18:58:32.421 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-16T18:58:32.830 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 18 kB 00:00 2024-08-16T18:58:33.043 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-08-16T18:58:33.926 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-08-16T18:58:37.794 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-08-16T18:58:38.744 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-16T18:58:39.307 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-16T18:58:40.032 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:40.032 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:40.040 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-16T18:58:40.041 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-16T18:58:40.042 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-16T18:58:40.042 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-16T18:58:40.044 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-16T18:58:40.046 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-16T18:58:40.047 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-16T18:58:40.110 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-08-16T18:58:40.130 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 6.6 k 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 896 k 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 145 k 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 1.5 M 2024-08-16T18:58:40.131 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 146 k 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 3.6 M 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 7.4 M 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 49 k 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 11 M 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 49 M 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 276 k 2024-08-16T18:58:40.132 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 765 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 225 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 27 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 18 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 32 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 751 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 127 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-16T18:58:40.133 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 188 k 2024-08-16T18:58:40.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 371 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 349 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 116 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 85 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.1 M 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 170 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-16T18:58:40.135 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.4 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.2 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 21 M 2024-08-16T18:58:40.136 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 32 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 2.4 M 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 249 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 4.7 M 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 17 M 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 16 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 25 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-16T18:58:40.137 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 163 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 506 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-16T18:58:40.138 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-16T18:58:40.139 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 45 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 136 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-16T18:58:40.140 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-16T18:58:40.141 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-16T18:58:40.142 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-16T18:58:40.143 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-16T18:58:40.144 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 12 Packages 2024-08-16T18:58:40.145 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:58:40.146 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 228 M 2024-08-16T18:58:40.146 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-08-16T18:58:40.653 INFO:teuthology.orchestra.run.smithi161.stdout:(1/160): ceph-19.1.0-1363.gd863d21c.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-08-16T18:58:40.887 INFO:teuthology.orchestra.run.smithi161.stdout:(2/160): ceph-fuse-19.1.0-1363.gd863d21c.el9.x8 3.8 MB/s | 896 kB 00:00 2024-08-16T18:58:40.987 INFO:teuthology.orchestra.run.smithi161.stdout:(3/160): ceph-base-19.1.0-1363.gd863d21c.el9.x8 11 MB/s | 5.4 MB 00:00 2024-08-16T18:58:41.013 INFO:teuthology.orchestra.run.smithi161.stdout:(4/160): ceph-immutable-object-cache-19.1.0-136 1.1 MB/s | 145 kB 00:00 2024-08-16T18:58:41.148 INFO:teuthology.orchestra.run.smithi161.stdout:(5/160): ceph-mds-19.1.0-1363.gd863d21c.el9.x86 15 MB/s | 2.4 MB 00:00 2024-08-16T18:58:41.205 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.206 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.218 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-16T18:58:41.218 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.220 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-16T18:58:41.220 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-16T18:58:41.222 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-16T18:58:41.225 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-16T18:58:41.225 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-16T18:58:41.254 INFO:teuthology.orchestra.run.smithi161.stdout:(6/160): ceph-mgr-19.1.0-1363.gd863d21c.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-08-16T18:58:41.295 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-08-16T18:58:41.324 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-08-16T18:58:41.324 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 6.6 k 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 896 k 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 145 k 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 1.5 M 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 146 k 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 3.6 M 2024-08-16T18:58:41.325 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 7.4 M 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 49 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 11 M 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 49 M 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 276 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 765 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 225 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 27 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 18 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-16T18:58:41.326 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 32 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 751 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 127 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 188 k 2024-08-16T18:58:41.327 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 371 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 349 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 116 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 85 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.1 M 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 170 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-16T18:58:41.328 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.4 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.2 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-16T18:58:41.329 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 21 M 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 32 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 2.4 M 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 249 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 4.7 M 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 17 M 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 16 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 25 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-16T18:58:41.330 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 163 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 506 k 2024-08-16T18:58:41.331 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-16T18:58:41.332 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 45 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 136 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-16T18:58:41.333 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-16T18:58:41.334 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-16T18:58:41.335 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-16T18:58:41.336 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-16T18:58:41.337 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-08-16T18:58:41.338 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-08-16T18:58:41.339 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 12 Packages 2024-08-16T18:58:41.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:58:41.339 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 228 M 2024-08-16T18:58:41.339 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-08-16T18:58:41.388 INFO:teuthology.orchestra.run.smithi161.stdout:(7/160): ceph-mon-19.1.0-1363.gd863d21c.el9.x86 20 MB/s | 4.7 MB 00:00 2024-08-16T18:58:41.533 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.534 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.545 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-16T18:58:41.546 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-16T18:58:41.547 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-16T18:58:41.548 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-16T18:58:41.549 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-16T18:58:41.552 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-16T18:58:41.552 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-16T18:58:41.617 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-08-16T18:58:41.633 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 6.6 k 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 896 k 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 145 k 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 1.5 M 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 146 k 2024-08-16T18:58:41.634 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 3.6 M 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 7.4 M 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 49 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 11 M 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 49 M 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 276 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 765 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 225 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 27 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 18 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-16T18:58:41.635 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 32 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 751 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 127 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 188 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 371 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 349 k 2024-08-16T18:58:41.636 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 116 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 85 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.1 M 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 170 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.4 M 2024-08-16T18:58:41.637 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 3.2 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 21 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 32 k 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 2.4 M 2024-08-16T18:58:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 249 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 4.7 M 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 17 M 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1363.gd863d21c.el9 ceph-noarch 16 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 25 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-16T18:58:41.639 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 163 k 2024-08-16T18:58:41.640 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-16T18:58:41.640 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-16T18:58:41.640 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-16T18:58:41.640 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-16T18:58:41.640 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 506 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 5.4 M 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-16T18:58:41.641 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-16T18:58:41.642 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 45 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.1.0-1363.gd863d21c.el9 ceph 136 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-16T18:58:41.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-16T18:58:41.644 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-16T18:58:41.645 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-16T18:58:41.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-16T18:58:41.647 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-16T18:58:41.648 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================= 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout:Install 148 Packages 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 12 Packages 2024-08-16T18:58:41.649 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:58:41.650 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 228 M 2024-08-16T18:58:41.650 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-08-16T18:58:41.805 INFO:teuthology.orchestra.run.smithi161.stdout:(8/160): ceph-common-19.1.0-1363.gd863d21c.el9. 16 MB/s | 21 MB 00:01 2024-08-16T18:58:41.856 INFO:teuthology.orchestra.run.smithi161.stdout:(9/160): ceph-selinux-19.1.0-1363.gd863d21c.el9 504 kB/s | 25 kB 00:00 2024-08-16T18:58:41.981 INFO:teuthology.orchestra.run.smithi161.stdout:(10/160): ceph-radosgw-19.1.0-1363.gd863d21c.el 18 MB/s | 11 MB 00:00 2024-08-16T18:58:42.040 INFO:teuthology.orchestra.run.smithi161.stdout:(11/160): cephfs-mirror-19.1.0-1363.gd863d21c.e 3.8 MB/s | 225 kB 00:00 2024-08-16T18:58:42.123 INFO:teuthology.orchestra.run.smithi161.stdout:(12/160): cephfs-shell-19.1.0-1363.gd863d21c.el 325 kB/s | 27 kB 00:00 2024-08-16T18:58:42.215 INFO:teuthology.orchestra.run.smithi161.stdout:(13/160): libcephfs-devel-19.1.0-1363.gd863d21c 346 kB/s | 32 kB 00:00 2024-08-16T18:58:42.324 INFO:teuthology.orchestra.run.smithi161.stdout:(14/160): libcephfs2-19.1.0-1363.gd863d21c.el9. 6.8 MB/s | 751 kB 00:00 2024-08-16T18:58:42.491 INFO:teuthology.orchestra.run.smithi161.stdout:(15/160): ceph-osd-19.1.0-1363.gd863d21c.el9.x8 14 MB/s | 17 MB 00:01 2024-08-16T18:58:42.491 INFO:teuthology.orchestra.run.smithi155.stdout:(1/160): ceph-19.1.0-1363.gd863d21c.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-08-16T18:58:42.512 INFO:teuthology.orchestra.run.smithi062.stdout:(1/160): ceph-19.1.0-1363.gd863d21c.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-08-16T18:58:42.516 INFO:teuthology.orchestra.run.smithi161.stdout:(16/160): libcephsqlite-19.1.0-1363.gd863d21c.e 851 kB/s | 163 kB 00:00 2024-08-16T18:58:42.541 INFO:teuthology.orchestra.run.smithi161.stdout:(17/160): librados-devel-19.1.0-1363.gd863d21c. 2.5 MB/s | 127 kB 00:00 2024-08-16T18:58:42.583 INFO:teuthology.orchestra.run.smithi161.stdout:(18/160): libradosstriper1-19.1.0-1363.gd863d21 7.4 MB/s | 506 kB 00:00 2024-08-16T18:58:42.659 INFO:teuthology.orchestra.run.smithi161.stdout:(19/160): python3-ceph-argparse-19.1.0-1363.gd8 603 kB/s | 45 kB 00:00 2024-08-16T18:58:42.751 INFO:teuthology.orchestra.run.smithi161.stdout:(20/160): python3-ceph-common-19.1.0-1363.gd863 1.4 MB/s | 136 kB 00:00 2024-08-16T18:58:42.826 INFO:teuthology.orchestra.run.smithi161.stdout:(21/160): librgw2-19.1.0-1363.gd863d21c.el9.x86 19 MB/s | 5.4 MB 00:00 2024-08-16T18:58:42.882 INFO:teuthology.orchestra.run.smithi161.stdout:(22/160): python3-cephfs-19.1.0-1363.gd863d21c. 1.4 MB/s | 188 kB 00:00 2024-08-16T18:58:42.909 INFO:teuthology.orchestra.run.smithi161.stdout:(23/160): python3-rados-19.1.0-1363.gd863d21c.e 4.3 MB/s | 371 kB 00:00 2024-08-16T18:58:42.935 INFO:teuthology.orchestra.run.smithi161.stdout:(24/160): python3-rbd-19.1.0-1363.gd863d21c.el9 6.4 MB/s | 349 kB 00:00 2024-08-16T18:58:42.960 INFO:teuthology.orchestra.run.smithi161.stdout:(25/160): python3-rgw-19.1.0-1363.gd863d21c.el9 2.3 MB/s | 116 kB 00:00 2024-08-16T18:58:42.978 INFO:teuthology.orchestra.run.smithi062.stdout:(2/160): ceph-fuse-19.1.0-1363.gd863d21c.el9.x8 1.9 MB/s | 896 kB 00:00 2024-08-16T18:58:42.985 INFO:teuthology.orchestra.run.smithi161.stdout:(26/160): rbd-fuse-19.1.0-1363.gd863d21c.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-16T18:58:42.999 INFO:teuthology.orchestra.run.smithi155.stdout:(2/160): ceph-fuse-19.1.0-1363.gd863d21c.el9.x8 1.7 MB/s | 896 kB 00:00 2024-08-16T18:58:43.052 INFO:teuthology.orchestra.run.smithi161.stdout:(27/160): rbd-nbd-19.1.0-1363.gd863d21c.el9.x86 2.5 MB/s | 170 kB 00:00 2024-08-16T18:58:43.119 INFO:teuthology.orchestra.run.smithi161.stdout:(28/160): rbd-mirror-19.1.0-1363.gd863d21c.el9. 20 MB/s | 3.1 MB 00:00 2024-08-16T18:58:43.137 INFO:teuthology.orchestra.run.smithi161.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1363.g 381 kB/s | 32 kB 00:00 2024-08-16T18:58:43.141 INFO:teuthology.orchestra.run.smithi155.stdout:(3/160): ceph-immutable-object-cache-19.1.0-136 1.0 MB/s | 145 kB 00:00 2024-08-16T18:58:43.187 INFO:teuthology.orchestra.run.smithi161.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1363.gd863d21 2.1 MB/s | 146 kB 00:00 2024-08-16T18:58:43.404 INFO:teuthology.orchestra.run.smithi161.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1363.gd863d 13 MB/s | 3.6 MB 00:00 2024-08-16T18:58:43.404 INFO:teuthology.orchestra.run.smithi062.stdout:(3/160): ceph-immutable-object-cache-19.1.0-136 341 kB/s | 145 kB 00:00 2024-08-16T18:58:43.487 INFO:teuthology.orchestra.run.smithi161.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1363.gd8 2.9 MB/s | 249 kB 00:00 2024-08-16T18:58:43.579 INFO:teuthology.orchestra.run.smithi161.stdout:(33/160): ceph-mgr-rook-19.1.0-1363.gd863d21c.e 539 kB/s | 49 kB 00:00 2024-08-16T18:58:43.671 INFO:teuthology.orchestra.run.smithi161.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1363.gd 180 kB/s | 16 kB 00:00 2024-08-16T18:58:43.780 INFO:teuthology.orchestra.run.smithi161.stdout:(35/160): ceph-mgr-diskprediction-local-19.1.0- 13 MB/s | 7.4 MB 00:00 2024-08-16T18:58:43.796 INFO:teuthology.orchestra.run.smithi062.stdout:(4/160): ceph-base-19.1.0-1363.gd863d21c.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-08-16T18:58:43.805 INFO:teuthology.orchestra.run.smithi161.stdout:(36/160): ceph-volume-19.1.0-1363.gd863d21c.el9 2.0 MB/s | 276 kB 00:00 2024-08-16T18:58:43.825 INFO:teuthology.orchestra.run.smithi155.stdout:(4/160): ceph-base-19.1.0-1363.gd863d21c.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-08-16T18:58:43.847 INFO:teuthology.orchestra.run.smithi161.stdout:(37/160): cephadm-19.1.0-1363.gd863d21c.el9.noa 11 MB/s | 765 kB 00:00 2024-08-16T18:58:43.873 INFO:teuthology.orchestra.run.smithi161.stdout:(38/160): cephfs-top-19.1.0-1363.gd863d21c.el9. 271 kB/s | 18 kB 00:00 2024-08-16T18:58:43.950 INFO:teuthology.orchestra.run.smithi155.stdout:(5/160): ceph-mgr-19.1.0-1363.gd863d21c.el9.x86 12 MB/s | 1.5 MB 00:00 2024-08-16T18:58:44.012 INFO:teuthology.orchestra.run.smithi062.stdout:(5/160): ceph-mgr-19.1.0-1363.gd863d21c.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-08-16T18:58:44.209 INFO:teuthology.orchestra.run.smithi155.stdout:(6/160): ceph-mon-19.1.0-1363.gd863d21c.el9.x86 18 MB/s | 4.7 MB 00:00 2024-08-16T18:58:44.298 INFO:teuthology.orchestra.run.smithi161.stdout:(39/160): ceph-test-19.1.0-1363.gd863d21c.el9.x 20 MB/s | 49 MB 00:02 2024-08-16T18:58:44.390 INFO:teuthology.orchestra.run.smithi161.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 147 kB/s | 80 kB 00:00 2024-08-16T18:58:44.415 INFO:teuthology.orchestra.run.smithi161.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-08-16T18:58:44.438 INFO:teuthology.orchestra.run.smithi062.stdout:(6/160): ceph-mds-19.1.0-1363.gd863d21c.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-08-16T18:58:44.465 INFO:teuthology.orchestra.run.smithi161.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-08-16T18:58:44.507 INFO:teuthology.orchestra.run.smithi161.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-08-16T18:58:44.533 INFO:teuthology.orchestra.run.smithi161.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-08-16T18:58:44.558 INFO:teuthology.orchestra.run.smithi161.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-16T18:58:44.583 INFO:teuthology.orchestra.run.smithi161.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-16T18:58:44.584 INFO:teuthology.orchestra.run.smithi155.stdout:(7/160): ceph-mds-19.1.0-1363.gd863d21c.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-08-16T18:58:44.608 INFO:teuthology.orchestra.run.smithi161.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-16T18:58:44.692 INFO:teuthology.orchestra.run.smithi161.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-08-16T18:58:44.730 INFO:teuthology.orchestra.run.smithi062.stdout:(7/160): ceph-mon-19.1.0-1363.gd863d21c.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-08-16T18:58:44.734 INFO:teuthology.orchestra.run.smithi161.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-08-16T18:58:44.784 INFO:teuthology.orchestra.run.smithi161.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-08-16T18:58:44.801 INFO:teuthology.orchestra.run.smithi161.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-08-16T18:58:44.826 INFO:teuthology.orchestra.run.smithi161.stdout:(52/160): boost-program-options-1.75.0-8.el9.x8 797 kB/s | 107 kB 00:00 2024-08-16T18:58:44.852 INFO:teuthology.orchestra.run.smithi161.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-08-16T18:58:44.868 INFO:teuthology.orchestra.run.smithi155.stdout:(8/160): ceph-osd-19.1.0-1363.gd863d21c.el9.x86 26 MB/s | 17 MB 00:00 2024-08-16T18:58:44.894 INFO:teuthology.orchestra.run.smithi161.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-08-16T18:58:44.910 INFO:teuthology.orchestra.run.smithi155.stdout:(9/160): ceph-selinux-19.1.0-1363.gd863d21c.el9 608 kB/s | 25 kB 00:00 2024-08-16T18:58:44.919 INFO:teuthology.orchestra.run.smithi161.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-08-16T18:58:44.944 INFO:teuthology.orchestra.run.smithi161.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-08-16T18:58:44.978 INFO:teuthology.orchestra.run.smithi161.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-16T18:58:45.003 INFO:teuthology.orchestra.run.smithi161.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-08-16T18:58:45.037 INFO:teuthology.orchestra.run.smithi161.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-08-16T18:58:45.087 INFO:teuthology.orchestra.run.smithi161.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-08-16T18:58:45.112 INFO:teuthology.orchestra.run.smithi161.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-08-16T18:58:45.138 INFO:teuthology.orchestra.run.smithi161.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-08-16T18:58:45.189 INFO:teuthology.orchestra.run.smithi161.stdout:(63/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.3 MB/s | 3.0 MB 00:00 2024-08-16T18:58:45.214 INFO:teuthology.orchestra.run.smithi161.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-08-16T18:58:45.241 INFO:teuthology.orchestra.run.smithi161.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 673 kB/s | 35 kB 00:00 2024-08-16T18:58:45.267 INFO:teuthology.orchestra.run.smithi161.stdout:(66/160): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-08-16T18:58:45.392 INFO:teuthology.orchestra.run.smithi161.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 30 MB/s | 4.4 MB 00:00 2024-08-16T18:58:45.433 INFO:teuthology.orchestra.run.smithi161.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-08-16T18:58:45.500 INFO:teuthology.orchestra.run.smithi161.stdout:(69/160): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-08-16T18:58:45.533 INFO:teuthology.orchestra.run.smithi161.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-08-16T18:58:45.559 INFO:teuthology.orchestra.run.smithi161.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-08-16T18:58:45.609 INFO:teuthology.orchestra.run.smithi161.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-08-16T18:58:45.626 INFO:teuthology.orchestra.run.smithi161.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-08-16T18:58:45.710 INFO:teuthology.orchestra.run.smithi161.stdout:(74/160): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-08-16T18:58:45.735 INFO:teuthology.orchestra.run.smithi161.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-08-16T18:58:45.760 INFO:teuthology.orchestra.run.smithi161.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-08-16T18:58:45.786 INFO:teuthology.orchestra.run.smithi161.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-16T18:58:45.811 INFO:teuthology.orchestra.run.smithi161.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-08-16T18:58:45.827 INFO:teuthology.orchestra.run.smithi161.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-08-16T18:58:45.853 INFO:teuthology.orchestra.run.smithi161.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-08-16T18:58:45.878 INFO:teuthology.orchestra.run.smithi161.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-08-16T18:58:45.970 INFO:teuthology.orchestra.run.smithi161.stdout:(82/160): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-08-16T18:58:45.995 INFO:teuthology.orchestra.run.smithi161.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-16T18:58:46.021 INFO:teuthology.orchestra.run.smithi161.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-16T18:58:46.046 INFO:teuthology.orchestra.run.smithi161.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-16T18:58:46.071 INFO:teuthology.orchestra.run.smithi161.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-08-16T18:58:46.096 INFO:teuthology.orchestra.run.smithi161.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-08-16T18:58:46.122 INFO:teuthology.orchestra.run.smithi161.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-16T18:58:46.147 INFO:teuthology.orchestra.run.smithi161.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-16T18:58:46.173 INFO:teuthology.orchestra.run.smithi161.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-16T18:58:46.198 INFO:teuthology.orchestra.run.smithi161.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-16T18:58:46.273 INFO:teuthology.orchestra.run.smithi161.stdout:(92/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-16T18:58:46.298 INFO:teuthology.orchestra.run.smithi161.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-08-16T18:58:46.323 INFO:teuthology.orchestra.run.smithi161.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-08-16T18:58:46.349 INFO:teuthology.orchestra.run.smithi161.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-16T18:58:46.399 INFO:teuthology.orchestra.run.smithi161.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-08-16T18:58:46.424 INFO:teuthology.orchestra.run.smithi161.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-08-16T18:58:46.426 INFO:teuthology.orchestra.run.smithi155.stdout:(10/160): ceph-test-19.1.0-1363.gd863d21c.el9.x 32 MB/s | 49 MB 00:01 2024-08-16T18:58:46.450 INFO:teuthology.orchestra.run.smithi161.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-16T18:58:46.475 INFO:teuthology.orchestra.run.smithi161.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-16T18:58:46.477 INFO:teuthology.orchestra.run.smithi155.stdout:(11/160): cephfs-mirror-19.1.0-1363.gd863d21c.e 4.4 MB/s | 225 kB 00:00 2024-08-16T18:58:46.500 INFO:teuthology.orchestra.run.smithi161.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-16T18:58:46.526 INFO:teuthology.orchestra.run.smithi161.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-16T18:58:46.527 INFO:teuthology.orchestra.run.smithi155.stdout:(12/160): cephfs-shell-19.1.0-1363.gd863d21c.el 544 kB/s | 27 kB 00:00 2024-08-16T18:58:46.551 INFO:teuthology.orchestra.run.smithi161.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-08-16T18:58:46.555 INFO:teuthology.orchestra.run.smithi062.stdout:(8/160): ceph-radosgw-19.1.0-1363.gd863d21c.el9 5.8 MB/s | 11 MB 00:01 2024-08-16T18:58:46.569 INFO:teuthology.orchestra.run.smithi155.stdout:(13/160): libcephfs-devel-19.1.0-1363.gd863d21c 767 kB/s | 32 kB 00:00 2024-08-16T18:58:46.576 INFO:teuthology.orchestra.run.smithi161.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-08-16T18:58:46.597 INFO:teuthology.orchestra.run.smithi062.stdout:(9/160): ceph-selinux-19.1.0-1363.gd863d21c.el9 607 kB/s | 25 kB 00:00 2024-08-16T18:58:46.628 INFO:teuthology.orchestra.run.smithi155.stdout:(14/160): libcephfs2-19.1.0-1363.gd863d21c.el9. 13 MB/s | 751 kB 00:00 2024-08-16T18:58:46.635 INFO:teuthology.orchestra.run.smithi161.stdout:(104/160): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-08-16T18:58:46.678 INFO:teuthology.orchestra.run.smithi155.stdout:(15/160): libcephsqlite-19.1.0-1363.gd863d21c.e 3.2 MB/s | 163 kB 00:00 2024-08-16T18:58:46.710 INFO:teuthology.orchestra.run.smithi161.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-08-16T18:58:46.720 INFO:teuthology.orchestra.run.smithi155.stdout:(16/160): librados-devel-19.1.0-1363.gd863d21c. 3.0 MB/s | 127 kB 00:00 2024-08-16T18:58:46.735 INFO:teuthology.orchestra.run.smithi161.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-08-16T18:58:46.769 INFO:teuthology.orchestra.run.smithi161.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-08-16T18:58:46.787 INFO:teuthology.orchestra.run.smithi155.stdout:(17/160): libradosstriper1-19.1.0-1363.gd863d21 7.4 MB/s | 506 kB 00:00 2024-08-16T18:58:46.794 INFO:teuthology.orchestra.run.smithi161.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-16T18:58:46.819 INFO:teuthology.orchestra.run.smithi161.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-16T18:58:46.845 INFO:teuthology.orchestra.run.smithi161.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-16T18:58:46.878 INFO:teuthology.orchestra.run.smithi161.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-16T18:58:47.004 INFO:teuthology.orchestra.run.smithi155.stdout:(18/160): librgw2-19.1.0-1363.gd863d21c.el9.x86 25 MB/s | 5.4 MB 00:00 2024-08-16T18:58:47.046 INFO:teuthology.orchestra.run.smithi155.stdout:(19/160): python3-ceph-argparse-19.1.0-1363.gd8 1.1 MB/s | 45 kB 00:00 2024-08-16T18:58:47.062 INFO:teuthology.orchestra.run.smithi161.stdout:(112/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-08-16T18:58:47.087 INFO:teuthology.orchestra.run.smithi155.stdout:(20/160): python3-ceph-common-19.1.0-1363.gd863 3.2 MB/s | 136 kB 00:00 2024-08-16T18:58:47.112 INFO:teuthology.orchestra.run.smithi161.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 839 kB 00:00 2024-08-16T18:58:47.138 INFO:teuthology.orchestra.run.smithi155.stdout:(21/160): python3-cephfs-19.1.0-1363.gd863d21c. 3.7 MB/s | 188 kB 00:00 2024-08-16T18:58:47.163 INFO:teuthology.orchestra.run.smithi161.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-16T18:58:47.188 INFO:teuthology.orchestra.run.smithi161.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-08-16T18:58:47.188 INFO:teuthology.orchestra.run.smithi155.stdout:(22/160): python3-rados-19.1.0-1363.gd863d21c.e 7.3 MB/s | 371 kB 00:00 2024-08-16T18:58:47.213 INFO:teuthology.orchestra.run.smithi161.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-08-16T18:58:47.239 INFO:teuthology.orchestra.run.smithi161.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-08-16T18:58:47.240 INFO:teuthology.orchestra.run.smithi155.stdout:(23/160): python3-rbd-19.1.0-1363.gd863d21c.el9 6.8 MB/s | 349 kB 00:00 2024-08-16T18:58:47.289 INFO:teuthology.orchestra.run.smithi155.stdout:(24/160): python3-rgw-19.1.0-1363.gd863d21c.el9 2.3 MB/s | 116 kB 00:00 2024-08-16T18:58:47.306 INFO:teuthology.orchestra.run.smithi161.stdout:(118/160): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-08-16T18:58:47.331 INFO:teuthology.orchestra.run.smithi161.stdout:(119/160): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-08-16T18:58:47.339 INFO:teuthology.orchestra.run.smithi155.stdout:(25/160): rbd-fuse-19.1.0-1363.gd863d21c.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-16T18:58:47.356 INFO:teuthology.orchestra.run.smithi161.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-08-16T18:58:47.381 INFO:teuthology.orchestra.run.smithi161.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-08-16T18:58:47.406 INFO:teuthology.orchestra.run.smithi161.stdout:(122/160): python3-google-auth-2.33.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-08-16T18:58:47.423 INFO:teuthology.orchestra.run.smithi161.stdout:(123/160): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-08-16T18:58:47.440 INFO:teuthology.orchestra.run.smithi161.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 317 kB/s | 11 kB 00:00 2024-08-16T18:58:47.466 INFO:teuthology.orchestra.run.smithi161.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-08-16T18:58:47.472 INFO:teuthology.orchestra.run.smithi062.stdout:(10/160): ceph-common-19.1.0-1363.gd863d21c.el9 4.1 MB/s | 21 MB 00:05 2024-08-16T18:58:47.473 INFO:teuthology.orchestra.run.smithi155.stdout:(26/160): rbd-mirror-19.1.0-1363.gd863d21c.el9. 23 MB/s | 3.1 MB 00:00 2024-08-16T18:58:47.499 INFO:teuthology.orchestra.run.smithi161.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-08-16T18:58:47.523 INFO:teuthology.orchestra.run.smithi155.stdout:(27/160): rbd-nbd-19.1.0-1363.gd863d21c.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-16T18:58:47.525 INFO:teuthology.orchestra.run.smithi161.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-08-16T18:58:47.550 INFO:teuthology.orchestra.run.smithi161.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-08-16T18:58:47.567 INFO:teuthology.orchestra.run.smithi062.stdout:(11/160): cephfs-mirror-19.1.0-1363.gd863d21c.e 2.3 MB/s | 225 kB 00:00 2024-08-16T18:58:47.575 INFO:teuthology.orchestra.run.smithi161.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-16T18:58:47.600 INFO:teuthology.orchestra.run.smithi161.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-16T18:58:47.614 INFO:teuthology.orchestra.run.smithi062.stdout:(12/160): cephfs-shell-19.1.0-1363.gd863d21c.el 579 kB/s | 27 kB 00:00 2024-08-16T18:58:47.634 INFO:teuthology.orchestra.run.smithi161.stdout:(131/160): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-08-16T18:58:47.656 INFO:teuthology.orchestra.run.smithi062.stdout:(13/160): libcephfs-devel-19.1.0-1363.gd863d21c 762 kB/s | 32 kB 00:00 2024-08-16T18:58:47.659 INFO:teuthology.orchestra.run.smithi161.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-08-16T18:58:47.685 INFO:teuthology.orchestra.run.smithi161.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-08-16T18:58:47.710 INFO:teuthology.orchestra.run.smithi161.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-08-16T18:58:47.715 INFO:teuthology.orchestra.run.smithi155.stdout:(28/160): ceph-common-19.1.0-1363.gd863d21c.el9 3.9 MB/s | 21 MB 00:05 2024-08-16T18:58:47.740 INFO:teuthology.orchestra.run.smithi155.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1363.g 147 kB/s | 32 kB 00:00 2024-08-16T18:58:47.752 INFO:teuthology.orchestra.run.smithi161.stdout:(135/160): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-08-16T18:58:47.773 INFO:teuthology.orchestra.run.smithi062.stdout:(14/160): libcephfs2-19.1.0-1363.gd863d21c.el9. 6.3 MB/s | 751 kB 00:00 2024-08-16T18:58:47.777 INFO:teuthology.orchestra.run.smithi161.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-08-16T18:58:47.791 INFO:teuthology.orchestra.run.smithi155.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1363.gd863d21 1.9 MB/s | 146 kB 00:00 2024-08-16T18:58:47.802 INFO:teuthology.orchestra.run.smithi161.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-08-16T18:58:47.828 INFO:teuthology.orchestra.run.smithi161.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-16T18:58:47.840 INFO:teuthology.orchestra.run.smithi062.stdout:(15/160): libcephsqlite-19.1.0-1363.gd863d21c.e 2.4 MB/s | 163 kB 00:00 2024-08-16T18:58:47.853 INFO:teuthology.orchestra.run.smithi161.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-16T18:58:47.878 INFO:teuthology.orchestra.run.smithi161.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-08-16T18:58:47.891 INFO:teuthology.orchestra.run.smithi062.stdout:(16/160): librados-devel-19.1.0-1363.gd863d21c. 2.5 MB/s | 127 kB 00:00 2024-08-16T18:58:47.899 INFO:teuthology.orchestra.run.smithi155.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1363.gd863d 22 MB/s | 3.6 MB 00:00 2024-08-16T18:58:47.903 INFO:teuthology.orchestra.run.smithi161.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-08-16T18:58:47.929 INFO:teuthology.orchestra.run.smithi161.stdout:(142/160): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-16T18:58:47.950 INFO:teuthology.orchestra.run.smithi155.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1363.gd8 4.9 MB/s | 249 kB 00:00 2024-08-16T18:58:47.954 INFO:teuthology.orchestra.run.smithi161.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-16T18:58:47.979 INFO:teuthology.orchestra.run.smithi161.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-16T18:58:47.982 INFO:teuthology.orchestra.run.smithi062.stdout:(17/160): libradosstriper1-19.1.0-1363.gd863d21 5.4 MB/s | 506 kB 00:00 2024-08-16T18:58:48.000 INFO:teuthology.orchestra.run.smithi155.stdout:(33/160): ceph-mgr-rook-19.1.0-1363.gd863d21c.e 989 kB/s | 49 kB 00:00 2024-08-16T18:58:48.004 INFO:teuthology.orchestra.run.smithi161.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-08-16T18:58:48.030 INFO:teuthology.orchestra.run.smithi161.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-16T18:58:48.050 INFO:teuthology.orchestra.run.smithi155.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1363.gd 331 kB/s | 16 kB 00:00 2024-08-16T18:58:48.055 INFO:teuthology.orchestra.run.smithi161.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-16T18:58:48.100 INFO:teuthology.orchestra.run.smithi155.stdout:(35/160): ceph-volume-19.1.0-1363.gd863d21c.el9 5.4 MB/s | 276 kB 00:00 2024-08-16T18:58:48.105 INFO:teuthology.orchestra.run.smithi161.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-16T18:58:48.159 INFO:teuthology.orchestra.run.smithi155.stdout:(36/160): cephadm-19.1.0-1363.gd863d21c.el9.noa 13 MB/s | 765 kB 00:00 2024-08-16T18:58:48.210 INFO:teuthology.orchestra.run.smithi155.stdout:(37/160): cephfs-top-19.1.0-1363.gd863d21c.el9. 364 kB/s | 18 kB 00:00 2024-08-16T18:58:48.247 INFO:teuthology.orchestra.run.smithi161.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 16 MB/s | 2.3 MB 00:00 2024-08-16T18:58:48.323 INFO:teuthology.orchestra.run.smithi161.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 24 MB/s | 1.8 MB 00:00 2024-08-16T18:58:48.381 INFO:teuthology.orchestra.run.smithi161.stdout:(151/160): librados2-19.1.0-1363.gd863d21c.el9. 9.7 MB/s | 3.4 MB 00:00 2024-08-16T18:58:48.415 INFO:teuthology.orchestra.run.smithi161.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 444 kB/s | 41 kB 00:00 2024-08-16T18:58:48.418 INFO:teuthology.orchestra.run.smithi155.stdout:(38/160): ceph-radosgw-19.1.0-1363.gd863d21c.el 2.7 MB/s | 11 MB 00:03 2024-08-16T18:58:48.465 INFO:teuthology.orchestra.run.smithi161.stdout:(153/160): librbd1-19.1.0-1363.gd863d21c.el9.x8 7.7 MB/s | 3.2 MB 00:00 2024-08-16T18:58:48.490 INFO:teuthology.orchestra.run.smithi161.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 488 kB/s | 37 kB 00:00 2024-08-16T18:58:48.532 INFO:teuthology.orchestra.run.smithi161.stdout:(155/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.6 MB/s | 247 kB 00:00 2024-08-16T18:58:48.558 INFO:teuthology.orchestra.run.smithi161.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.2 MB/s | 207 kB 00:00 2024-08-16T18:58:48.583 INFO:teuthology.orchestra.run.smithi062.stdout:(18/160): ceph-osd-19.1.0-1363.gd863d21c.el9.x8 4.1 MB/s | 17 MB 00:04 2024-08-16T18:58:48.583 INFO:teuthology.orchestra.run.smithi161.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.7 MB/s | 260 kB 00:00 2024-08-16T18:58:48.634 INFO:teuthology.orchestra.run.smithi062.stdout:(19/160): python3-ceph-argparse-19.1.0-1363.gd8 909 kB/s | 45 kB 00:00 2024-08-16T18:58:48.641 INFO:teuthology.orchestra.run.smithi161.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-08-16T18:58:48.643 INFO:teuthology.orchestra.run.smithi155.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 184 kB/s | 80 kB 00:00 2024-08-16T18:58:48.675 INFO:teuthology.orchestra.run.smithi161.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-08-16T18:58:48.683 INFO:teuthology.orchestra.run.smithi062.stdout:(20/160): python3-ceph-common-19.1.0-1363.gd863 2.7 MB/s | 136 kB 00:00 2024-08-16T18:58:48.717 INFO:teuthology.orchestra.run.smithi161.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-08-16T18:58:48.722 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-08-16T18:58:48.722 INFO:teuthology.orchestra.run.smithi161.stdout:Total 27 MB/s | 228 MB 00:08 2024-08-16T18:58:48.750 INFO:teuthology.orchestra.run.smithi062.stdout:(21/160): python3-cephfs-19.1.0-1363.gd863d21c. 2.8 MB/s | 188 kB 00:00 2024-08-16T18:58:48.834 INFO:teuthology.orchestra.run.smithi062.stdout:(22/160): librgw2-19.1.0-1363.gd863d21c.el9.x86 6.3 MB/s | 5.4 MB 00:00 2024-08-16T18:58:48.877 INFO:teuthology.orchestra.run.smithi155.stdout:(40/160): ceph-mgr-diskprediction-local-19.1.0- 6.8 MB/s | 7.4 MB 00:01 2024-08-16T18:58:48.884 INFO:teuthology.orchestra.run.smithi062.stdout:(23/160): python3-rados-19.1.0-1363.gd863d21c.e 2.7 MB/s | 371 kB 00:00 2024-08-16T18:58:48.910 INFO:teuthology.orchestra.run.smithi155.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 732 kB/s | 195 kB 00:00 2024-08-16T18:58:48.934 INFO:teuthology.orchestra.run.smithi062.stdout:(24/160): python3-rbd-19.1.0-1363.gd863d21c.el9 3.4 MB/s | 349 kB 00:00 2024-08-16T18:58:48.960 INFO:teuthology.orchestra.run.smithi062.stdout:(25/160): python3-rgw-19.1.0-1363.gd863d21c.el9 1.5 MB/s | 116 kB 00:00 2024-08-16T18:58:48.985 INFO:teuthology.orchestra.run.smithi062.stdout:(26/160): rbd-fuse-19.1.0-1363.gd863d21c.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-16T18:58:49.027 INFO:teuthology.orchestra.run.smithi062.stdout:(27/160): rbd-nbd-19.1.0-1363.gd863d21c.el9.x86 4.0 MB/s | 170 kB 00:00 2024-08-16T18:58:49.077 INFO:teuthology.orchestra.run.smithi062.stdout:(28/160): ceph-grafana-dashboards-19.1.0-1363.g 637 kB/s | 32 kB 00:00 2024-08-16T18:58:49.127 INFO:teuthology.orchestra.run.smithi062.stdout:(29/160): ceph-mgr-cephadm-19.1.0-1363.gd863d21 2.9 MB/s | 146 kB 00:00 2024-08-16T18:58:49.177 INFO:teuthology.orchestra.run.smithi155.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-08-16T18:58:49.277 INFO:teuthology.orchestra.run.smithi155.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-08-16T18:58:49.353 INFO:teuthology.orchestra.run.smithi062.stdout:(30/160): rbd-mirror-19.1.0-1363.gd863d21c.el9. 7.9 MB/s | 3.1 MB 00:00 2024-08-16T18:58:49.394 INFO:teuthology.orchestra.run.smithi155.stdout:(44/160): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-08-16T18:58:49.436 INFO:teuthology.orchestra.run.smithi155.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-08-16T18:58:49.461 INFO:teuthology.orchestra.run.smithi155.stdout:(46/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 433 kB/s | 253 kB 00:00 2024-08-16T18:58:49.503 INFO:teuthology.orchestra.run.smithi155.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-08-16T18:58:49.545 INFO:teuthology.orchestra.run.smithi155.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-16T18:58:49.645 INFO:teuthology.orchestra.run.smithi062.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1363.gd863d 6.9 MB/s | 3.6 MB 00:00 2024-08-16T18:58:49.687 INFO:teuthology.orchestra.run.smithi155.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 754 kB/s | 107 kB 00:00 2024-08-16T18:58:49.712 INFO:teuthology.orchestra.run.smithi062.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1363.gd8 3.7 MB/s | 249 kB 00:00 2024-08-16T18:58:49.729 INFO:teuthology.orchestra.run.smithi155.stdout:(50/160): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-08-16T18:58:49.754 INFO:teuthology.orchestra.run.smithi062.stdout:(33/160): ceph-mgr-rook-19.1.0-1363.gd863d21c.e 1.2 MB/s | 49 kB 00:00 2024-08-16T18:58:49.754 INFO:teuthology.orchestra.run.smithi155.stdout:(51/160): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-08-16T18:58:49.787 INFO:teuthology.orchestra.run.smithi062.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1363.gd 494 kB/s | 16 kB 00:00 2024-08-16T18:58:49.813 INFO:teuthology.orchestra.run.smithi155.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-08-16T18:58:49.839 INFO:teuthology.orchestra.run.smithi155.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-08-16T18:58:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:(35/160): ceph-volume-19.1.0-1363.gd863d21c.el9 3.6 MB/s | 276 kB 00:00 2024-08-16T18:58:49.864 INFO:teuthology.orchestra.run.smithi155.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-08-16T18:58:49.918 INFO:teuthology.orchestra.run.smithi155.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 231 kB/s | 18 kB 00:00 2024-08-16T18:58:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-08-16T18:58:49.971 INFO:teuthology.orchestra.run.smithi062.stdout:(36/160): cephadm-19.1.0-1363.gd863d21c.el9.noa 6.9 MB/s | 765 kB 00:00 2024-08-16T18:58:49.973 INFO:teuthology.orchestra.run.smithi155.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.8 MB/s | 160 kB 00:00 2024-08-16T18:58:49.975 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-08-16T18:58:49.998 INFO:teuthology.orchestra.run.smithi155.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-08-16T18:58:50.013 INFO:teuthology.orchestra.run.smithi062.stdout:(37/160): cephfs-top-19.1.0-1363.gd863d21c.el9. 440 kB/s | 18 kB 00:00 2024-08-16T18:58:50.132 INFO:teuthology.orchestra.run.smithi155.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-08-16T18:58:50.164 INFO:teuthology.orchestra.run.smithi062.stdout:(38/160): ceph-mgr-diskprediction-local-19.1.0- 9.1 MB/s | 7.4 MB 00:00 2024-08-16T18:58:50.168 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-08-16T18:58:50.168 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-08-16T18:58:50.182 INFO:teuthology.orchestra.run.smithi155.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-08-16T18:58:50.189 INFO:teuthology.orchestra.run.smithi062.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-08-16T18:58:50.207 INFO:teuthology.orchestra.run.smithi155.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-08-16T18:58:50.233 INFO:teuthology.orchestra.run.smithi155.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-08-16T18:58:50.266 INFO:teuthology.orchestra.run.smithi062.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-08-16T18:58:50.291 INFO:teuthology.orchestra.run.smithi155.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-08-16T18:58:50.306 INFO:teuthology.orchestra.run.smithi062.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-08-16T18:58:50.317 INFO:teuthology.orchestra.run.smithi155.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-08-16T18:58:50.331 INFO:teuthology.orchestra.run.smithi062.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-08-16T18:58:50.342 INFO:teuthology.orchestra.run.smithi155.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-08-16T18:58:50.382 INFO:teuthology.orchestra.run.smithi062.stdout:(43/160): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-08-16T18:58:50.392 INFO:teuthology.orchestra.run.smithi155.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-08-16T18:58:50.407 INFO:teuthology.orchestra.run.smithi062.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-08-16T18:58:50.432 INFO:teuthology.orchestra.run.smithi062.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-16T18:58:50.458 INFO:teuthology.orchestra.run.smithi062.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-16T18:58:50.483 INFO:teuthology.orchestra.run.smithi062.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-16T18:58:50.600 INFO:teuthology.orchestra.run.smithi062.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-08-16T18:58:50.641 INFO:teuthology.orchestra.run.smithi062.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-08-16T18:58:50.675 INFO:teuthology.orchestra.run.smithi062.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-08-16T18:58:50.701 INFO:teuthology.orchestra.run.smithi062.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-08-16T18:58:50.734 INFO:teuthology.orchestra.run.smithi062.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-08-16T18:58:50.759 INFO:teuthology.orchestra.run.smithi062.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-08-16T18:58:50.810 INFO:teuthology.orchestra.run.smithi062.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-08-16T18:58:50.868 INFO:teuthology.orchestra.run.smithi155.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-08-16T18:58:50.868 INFO:teuthology.orchestra.run.smithi062.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-08-16T18:58:50.893 INFO:teuthology.orchestra.run.smithi062.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-08-16T18:58:50.936 INFO:teuthology.orchestra.run.smithi062.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-08-16T18:58:50.961 INFO:teuthology.orchestra.run.smithi062.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-08-16T18:58:51.011 INFO:teuthology.orchestra.run.smithi062.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-08-16T18:58:51.045 INFO:teuthology.orchestra.run.smithi062.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-08-16T18:58:51.070 INFO:teuthology.orchestra.run.smithi062.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 432 kB/s | 25 kB 00:00 2024-08-16T18:58:51.093 INFO:teuthology.orchestra.run.smithi155.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-08-16T18:58:51.096 INFO:teuthology.orchestra.run.smithi062.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-08-16T18:58:51.121 INFO:teuthology.orchestra.run.smithi062.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-08-16T18:58:51.154 INFO:teuthology.orchestra.run.smithi062.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-08-16T18:58:51.171 INFO:teuthology.orchestra.run.smithi062.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-08-16T18:58:51.269 INFO:teuthology.orchestra.run.smithi155.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 2.2 MB/s | 911 kB 00:00 2024-08-16T18:58:51.280 INFO:teuthology.orchestra.run.smithi062.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-08-16T18:58:51.293 INFO:teuthology.orchestra.run.smithi155.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 666 kB/s | 133 kB 00:00 2024-08-16T18:58:51.418 INFO:teuthology.orchestra.run.smithi155.stdout:(71/160): openblas-openmp-0.3.21-2.el9.x86_64.r 4.3 MB/s | 4.7 MB 00:01 2024-08-16T18:58:51.469 INFO:teuthology.orchestra.run.smithi155.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-08-16T18:58:51.502 INFO:teuthology.orchestra.run.smithi155.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-08-16T18:58:51.661 INFO:teuthology.orchestra.run.smithi155.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 387 kB/s | 61 kB 00:00 2024-08-16T18:58:51.705 INFO:teuthology.orchestra.run.smithi062.stdout:(67/160): ceph-test-19.1.0-1363.gd863d21c.el9.x 9.5 MB/s | 49 MB 00:05 2024-08-16T18:58:51.756 INFO:teuthology.orchestra.run.smithi062.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-08-16T18:58:51.769 INFO:teuthology.orchestra.run.smithi155.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-08-16T18:58:51.823 INFO:teuthology.orchestra.run.smithi062.stdout:(69/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-08-16T18:58:51.848 INFO:teuthology.orchestra.run.smithi062.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-08-16T18:58:51.853 INFO:teuthology.orchestra.run.smithi155.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-08-16T18:58:51.903 INFO:teuthology.orchestra.run.smithi155.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-08-16T18:58:51.915 INFO:teuthology.orchestra.run.smithi062.stdout:(71/160): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-08-16T18:58:51.953 INFO:teuthology.orchestra.run.smithi155.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 385 kB/s | 19 kB 00:00 2024-08-16T18:58:51.992 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-08-16T18:58:51.992 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-08-16T18:58:51.998 INFO:teuthology.orchestra.run.smithi062.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-08-16T18:58:52.004 INFO:teuthology.orchestra.run.smithi155.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-16T18:58:52.023 INFO:teuthology.orchestra.run.smithi062.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-08-16T18:58:52.054 INFO:teuthology.orchestra.run.smithi155.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-08-16T18:58:52.057 INFO:teuthology.orchestra.run.smithi062.stdout:(74/160): postgresql-contrib-13.14-1.el9.x86_64 2.5 MB/s | 911 kB 00:00 2024-08-16T18:58:52.074 INFO:teuthology.orchestra.run.smithi062.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-16T18:58:52.175 INFO:teuthology.orchestra.run.smithi062.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-08-16T18:58:52.199 INFO:teuthology.orchestra.run.smithi062.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-08-16T18:58:52.225 INFO:teuthology.orchestra.run.smithi062.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-08-16T18:58:52.242 INFO:teuthology.orchestra.run.smithi062.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-08-16T18:58:52.259 INFO:teuthology.orchestra.run.smithi062.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 19 kB 00:00 2024-08-16T18:58:52.284 INFO:teuthology.orchestra.run.smithi062.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-08-16T18:58:52.310 INFO:teuthology.orchestra.run.smithi062.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-08-16T18:58:52.335 INFO:teuthology.orchestra.run.smithi062.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-08-16T18:58:52.385 INFO:teuthology.orchestra.run.smithi062.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-08-16T18:58:52.410 INFO:teuthology.orchestra.run.smithi062.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-08-16T18:58:52.427 INFO:teuthology.orchestra.run.smithi062.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-16T18:58:52.469 INFO:teuthology.orchestra.run.smithi062.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-08-16T18:58:52.494 INFO:teuthology.orchestra.run.smithi062.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-08-16T18:58:52.519 INFO:teuthology.orchestra.run.smithi062.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-16T18:58:52.553 INFO:teuthology.orchestra.run.smithi062.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-08-16T18:58:52.579 INFO:teuthology.orchestra.run.smithi062.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 910 kB/s | 54 kB 00:00 2024-08-16T18:58:52.646 INFO:teuthology.orchestra.run.smithi062.stdout:(92/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-08-16T18:58:52.671 INFO:teuthology.orchestra.run.smithi062.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-08-16T18:58:52.705 INFO:teuthology.orchestra.run.smithi062.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-08-16T18:58:52.738 INFO:teuthology.orchestra.run.smithi062.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-08-16T18:58:52.763 INFO:teuthology.orchestra.run.smithi062.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-08-16T18:58:52.789 INFO:teuthology.orchestra.run.smithi062.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-08-16T18:58:52.831 INFO:teuthology.orchestra.run.smithi062.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-08-16T18:58:52.856 INFO:teuthology.orchestra.run.smithi062.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-08-16T18:58:52.863 INFO:teuthology.orchestra.run.smithi155.stdout:(81/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-08-16T18:58:52.889 INFO:teuthology.orchestra.run.smithi062.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-08-16T18:58:52.918 INFO:teuthology.orchestra.run.smithi062.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-16T18:58:52.948 INFO:teuthology.orchestra.run.smithi062.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-08-16T18:58:52.971 INFO:teuthology.orchestra.run.smithi155.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-08-16T18:58:53.055 INFO:teuthology.orchestra.run.smithi155.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-08-16T18:58:53.105 INFO:teuthology.orchestra.run.smithi155.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-16T18:58:53.132 INFO:teuthology.orchestra.run.smithi062.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-08-16T18:58:53.165 INFO:teuthology.orchestra.run.smithi062.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-08-16T18:58:53.180 INFO:teuthology.orchestra.run.smithi155.stdout:(85/160): postgresql-server-13.14-1.el9.x86_64. 3.0 MB/s | 5.8 MB 00:01 2024-08-16T18:58:53.206 INFO:teuthology.orchestra.run.smithi155.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-08-16T18:58:53.266 INFO:teuthology.orchestra.run.smithi062.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-08-16T18:58:53.283 INFO:teuthology.orchestra.run.smithi062.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-08-16T18:58:53.306 INFO:teuthology.orchestra.run.smithi155.stdout:(87/160): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-08-16T18:58:53.316 INFO:teuthology.orchestra.run.smithi062.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-08-16T18:58:53.332 INFO:teuthology.orchestra.run.smithi155.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-08-16T18:58:53.333 INFO:teuthology.orchestra.run.smithi062.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-08-16T18:58:53.357 INFO:teuthology.orchestra.run.smithi155.stdout:(89/160): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-08-16T18:58:53.358 INFO:teuthology.orchestra.run.smithi062.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-16T18:58:53.382 INFO:teuthology.orchestra.run.smithi155.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-16T18:58:53.392 INFO:teuthology.orchestra.run.smithi062.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-16T18:58:53.407 INFO:teuthology.orchestra.run.smithi155.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-16T18:58:53.426 INFO:teuthology.orchestra.run.smithi062.stdout:(111/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-08-16T18:58:53.432 INFO:teuthology.orchestra.run.smithi155.stdout:(92/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-08-16T18:58:53.451 INFO:teuthology.orchestra.run.smithi062.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-08-16T18:58:53.457 INFO:teuthology.orchestra.run.smithi155.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-08-16T18:58:53.483 INFO:teuthology.orchestra.run.smithi155.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-08-16T18:58:53.534 INFO:teuthology.orchestra.run.smithi155.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-08-16T18:58:53.534 INFO:teuthology.orchestra.run.smithi062.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 9.7 MB/s | 548 kB 00:00 2024-08-16T18:58:53.560 INFO:teuthology.orchestra.run.smithi062.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-16T18:58:53.567 INFO:teuthology.orchestra.run.smithi155.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-08-16T18:58:53.585 INFO:teuthology.orchestra.run.smithi062.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-16T18:58:53.592 INFO:teuthology.orchestra.run.smithi155.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-08-16T18:58:53.610 INFO:teuthology.orchestra.run.smithi062.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-08-16T18:58:53.642 INFO:teuthology.orchestra.run.smithi155.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-16T18:58:53.677 INFO:teuthology.orchestra.run.smithi062.stdout:(117/160): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-16T18:58:53.703 INFO:teuthology.orchestra.run.smithi062.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-08-16T18:58:53.709 INFO:teuthology.orchestra.run.smithi155.stdout:(99/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.4 MB/s | 96 kB 00:00 2024-08-16T18:58:53.734 INFO:teuthology.orchestra.run.smithi155.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-08-16T18:58:53.736 INFO:teuthology.orchestra.run.smithi062.stdout:(119/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-08-16T18:58:53.770 INFO:teuthology.orchestra.run.smithi062.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-08-16T18:58:53.785 INFO:teuthology.orchestra.run.smithi155.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-08-16T18:58:53.795 INFO:teuthology.orchestra.run.smithi062.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-08-16T18:58:53.821 INFO:teuthology.orchestra.run.smithi062.stdout:(122/160): python3-google-auth-2.33.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-08-16T18:58:53.843 INFO:teuthology.orchestra.run.smithi155.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 476 kB/s | 64 kB 00:00 2024-08-16T18:58:53.846 INFO:teuthology.orchestra.run.smithi062.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-08-16T18:58:53.871 INFO:teuthology.orchestra.run.smithi062.stdout:(124/160): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-08-16T18:58:53.896 INFO:teuthology.orchestra.run.smithi062.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-16T18:58:53.921 INFO:teuthology.orchestra.run.smithi062.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-08-16T18:58:53.950 INFO:teuthology.orchestra.run.smithi062.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 364 kB/s | 19 kB 00:00 2024-08-16T18:58:53.969 INFO:teuthology.orchestra.run.smithi155.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-08-16T18:58:53.972 INFO:teuthology.orchestra.run.smithi062.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-08-16T18:58:53.997 INFO:teuthology.orchestra.run.smithi062.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-16T18:58:54.031 INFO:teuthology.orchestra.run.smithi062.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-08-16T18:58:54.056 INFO:teuthology.orchestra.run.smithi062.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-08-16T18:58:54.061 INFO:teuthology.orchestra.run.smithi155.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-08-16T18:58:54.098 INFO:teuthology.orchestra.run.smithi062.stdout:(132/160): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-08-16T18:58:54.103 INFO:teuthology.orchestra.run.smithi155.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-08-16T18:58:54.128 INFO:teuthology.orchestra.run.smithi155.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-08-16T18:58:54.140 INFO:teuthology.orchestra.run.smithi062.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 5.4 MB/s | 1.0 MB 00:00 2024-08-16T18:58:54.165 INFO:teuthology.orchestra.run.smithi062.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-08-16T18:58:54.191 INFO:teuthology.orchestra.run.smithi062.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-08-16T18:58:54.216 INFO:teuthology.orchestra.run.smithi062.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-16T18:58:54.241 INFO:teuthology.orchestra.run.smithi062.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-08-16T18:58:54.262 INFO:teuthology.orchestra.run.smithi155.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-08-16T18:58:54.266 INFO:teuthology.orchestra.run.smithi062.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-08-16T18:58:54.287 INFO:teuthology.orchestra.run.smithi155.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-16T18:58:54.292 INFO:teuthology.orchestra.run.smithi062.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-08-16T18:58:54.312 INFO:teuthology.orchestra.run.smithi155.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-16T18:58:54.317 INFO:teuthology.orchestra.run.smithi062.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-08-16T18:58:54.337 INFO:teuthology.orchestra.run.smithi155.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-16T18:58:54.342 INFO:teuthology.orchestra.run.smithi062.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-16T18:58:54.363 INFO:teuthology.orchestra.run.smithi155.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-08-16T18:58:54.367 INFO:teuthology.orchestra.run.smithi062.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-08-16T18:58:54.393 INFO:teuthology.orchestra.run.smithi062.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-16T18:58:54.404 INFO:teuthology.orchestra.run.smithi155.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-16T18:58:54.418 INFO:teuthology.orchestra.run.smithi062.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-16T18:58:54.440 INFO:teuthology.orchestra.run.smithi155.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 15 MB/s | 548 kB 00:00 2024-08-16T18:58:54.443 INFO:teuthology.orchestra.run.smithi062.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-08-16T18:58:54.466 INFO:teuthology.orchestra.run.smithi155.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-16T18:58:54.469 INFO:teuthology.orchestra.run.smithi062.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-16T18:58:54.491 INFO:teuthology.orchestra.run.smithi155.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-16T18:58:54.502 INFO:teuthology.orchestra.run.smithi062.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-08-16T18:58:54.516 INFO:teuthology.orchestra.run.smithi155.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-08-16T18:58:54.550 INFO:teuthology.orchestra.run.smithi155.stdout:(117/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-08-16T18:58:54.561 INFO:teuthology.orchestra.run.smithi062.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-08-16T18:58:54.600 INFO:teuthology.orchestra.run.smithi155.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-08-16T18:58:54.634 INFO:teuthology.orchestra.run.smithi155.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.6 MB/s | 303 kB 00:00 2024-08-16T18:58:54.667 INFO:teuthology.orchestra.run.smithi155.stdout:(120/160): python3-google-auth-2.33.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-08-16T18:58:54.693 INFO:teuthology.orchestra.run.smithi155.stdout:(121/160): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-08-16T18:58:54.726 INFO:teuthology.orchestra.run.smithi155.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-16T18:58:54.728 INFO:teuthology.orchestra.run.smithi062.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-08-16T18:58:54.752 INFO:teuthology.orchestra.run.smithi155.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-08-16T18:58:54.777 INFO:teuthology.orchestra.run.smithi155.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-16T18:58:54.802 INFO:teuthology.orchestra.run.smithi155.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-08-16T18:58:54.803 INFO:teuthology.orchestra.run.smithi062.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 24 MB/s | 1.8 MB 00:00 2024-08-16T18:58:54.827 INFO:teuthology.orchestra.run.smithi155.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-08-16T18:58:54.853 INFO:teuthology.orchestra.run.smithi155.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-08-16T18:58:54.862 INFO:teuthology.orchestra.run.smithi062.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 690 kB/s | 41 kB 00:00 2024-08-16T18:58:54.903 INFO:teuthology.orchestra.run.smithi155.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-08-16T18:58:54.920 INFO:teuthology.orchestra.run.smithi062.stdout:(152/160): librados2-19.1.0-1363.gd863d21c.el9. 7.6 MB/s | 3.4 MB 00:00 2024-08-16T18:58:54.928 INFO:teuthology.orchestra.run.smithi155.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-16T18:58:54.946 INFO:teuthology.orchestra.run.smithi062.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.9 MB/s | 247 kB 00:00 2024-08-16T18:58:54.954 INFO:teuthology.orchestra.run.smithi155.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-16T18:58:54.973 INFO:teuthology.orchestra.run.smithi062.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 732 kB/s | 37 kB 00:00 2024-08-16T18:58:54.979 INFO:teuthology.orchestra.run.smithi155.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-16T18:58:55.013 INFO:teuthology.orchestra.run.smithi155.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-16T18:58:55.021 INFO:teuthology.orchestra.run.smithi062.stdout:(155/160): librbd1-19.1.0-1363.gd863d21c.el9.x8 6.1 MB/s | 3.2 MB 00:00 2024-08-16T18:58:55.046 INFO:teuthology.orchestra.run.smithi155.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-08-16T18:58:55.047 INFO:teuthology.orchestra.run.smithi062.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.0 MB/s | 207 kB 00:00 2024-08-16T18:58:55.071 INFO:teuthology.orchestra.run.smithi155.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-08-16T18:58:55.080 INFO:teuthology.orchestra.run.smithi062.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-08-16T18:58:55.097 INFO:teuthology.orchestra.run.smithi155.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-16T18:58:55.113 INFO:teuthology.orchestra.run.smithi062.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-08-16T18:58:55.122 INFO:teuthology.orchestra.run.smithi155.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-16T18:58:55.147 INFO:teuthology.orchestra.run.smithi155.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-16T18:58:55.172 INFO:teuthology.orchestra.run.smithi062.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-08-16T18:58:55.173 INFO:teuthology.orchestra.run.smithi155.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-16T18:58:55.198 INFO:teuthology.orchestra.run.smithi155.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-16T18:58:55.215 INFO:teuthology.orchestra.run.smithi155.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-08-16T18:58:55.240 INFO:teuthology.orchestra.run.smithi155.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-16T18:58:55.266 INFO:teuthology.orchestra.run.smithi155.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-16T18:58:55.272 INFO:teuthology.orchestra.run.smithi062.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-08-16T18:58:55.279 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-08-16T18:58:55.279 INFO:teuthology.orchestra.run.smithi062.stdout:Total 16 MB/s | 228 MB 00:13 2024-08-16T18:58:55.291 INFO:teuthology.orchestra.run.smithi155.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-16T18:58:55.333 INFO:teuthology.orchestra.run.smithi155.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-08-16T18:58:55.358 INFO:teuthology.orchestra.run.smithi155.stdout:(145/160): re2-20211101-3.el9.x86_64.rpm 7.6 MB/s | 192 kB 00:00 2024-08-16T18:58:55.383 INFO:teuthology.orchestra.run.smithi155.stdout:(146/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-08-16T18:58:55.459 INFO:teuthology.orchestra.run.smithi155.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-16T18:58:55.634 INFO:teuthology.orchestra.run.smithi155.stdout:(148/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-08-16T18:58:55.768 INFO:teuthology.orchestra.run.smithi155.stdout:(149/160): librbd1-19.1.0-1363.gd863d21c.el9.x8 10 MB/s | 3.2 MB 00:00 2024-08-16T18:58:56.143 INFO:teuthology.orchestra.run.smithi155.stdout:(150/160): librados2-19.1.0-1363.gd863d21c.el9. 4.5 MB/s | 3.4 MB 00:00 2024-08-16T18:58:56.410 INFO:teuthology.orchestra.run.smithi155.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 153 kB/s | 41 kB 00:00 2024-08-16T18:58:56.544 INFO:teuthology.orchestra.run.smithi155.stdout:(152/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.3 MB/s | 1.8 MB 00:00 2024-08-16T18:58:56.585 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-08-16T18:58:56.685 INFO:teuthology.orchestra.run.smithi155.stdout:(153/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-08-16T18:58:56.710 INFO:teuthology.orchestra.run.smithi155.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 220 kB/s | 37 kB 00:00 2024-08-16T18:58:56.794 INFO:teuthology.orchestra.run.smithi155.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-08-16T18:58:56.810 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-08-16T18:58:56.810 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-08-16T18:58:56.819 INFO:teuthology.orchestra.run.smithi155.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-08-16T18:58:56.845 INFO:teuthology.orchestra.run.smithi155.stdout:(157/160): NetworkManager-tui-1.48.2-2.el9.x86_ 569 kB/s | 247 kB 00:00 2024-08-16T18:58:56.920 INFO:teuthology.orchestra.run.smithi155.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-08-16T18:58:57.337 INFO:teuthology.orchestra.run.smithi155.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 2.2 MB/s | 1.1 MB 00:00 2024-08-16T18:58:58.268 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-08-16T18:58:58.329 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-16T18:58:58.367 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-16T18:58:58.395 INFO:teuthology.orchestra.run.smithi155.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-08-16T18:58:58.401 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-08-16T18:58:58.402 INFO:teuthology.orchestra.run.smithi155.stdout:Total 14 MB/s | 228 MB 00:16 2024-08-16T18:58:58.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-16T18:58:58.455 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-16T18:58:58.480 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-16T18:58:58.522 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-16T18:58:58.853 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-16T18:58:58.965 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:58:59.071 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-08-16T18:58:59.072 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-08-16T18:58:59.086 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:58:59.116 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:58:59.194 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:58:59.220 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:58:59.253 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-16T18:58:59.443 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-16T18:58:59.492 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-16T18:58:59.521 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:58:59.624 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:58:59.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:58:59.648 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-08-16T18:58:59.709 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:58:59.731 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:58:59.750 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:58:59.845 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-08-16T18:58:59.845 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-08-16T18:58:59.906 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:00.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:00.195 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-16T18:59:00.364 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:00.407 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:00.505 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:00.601 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:00.812 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:00.926 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-16T18:59:01.028 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-16T18:59:01.064 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-16T18:59:01.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-16T18:59:01.165 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-16T18:59:01.217 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-16T18:59:01.257 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-16T18:59:01.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-16T18:59:01.341 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-16T18:59:01.369 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-16T18:59:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-16T18:59:01.491 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-16T18:59:01.522 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-16T18:59:01.557 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-16T18:59:01.633 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-16T18:59:01.668 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 36/172 2024-08-16T18:59:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-08-16T18:59:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-08-16T18:59:01.713 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 37/172 2024-08-16T18:59:01.740 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 38/172 2024-08-16T18:59:01.809 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-16T18:59:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-16T18:59:01.928 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-16T18:59:02.062 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-16T18:59:02.115 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-16T18:59:02.643 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-16T18:59:02.719 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-16T18:59:02.749 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-16T18:59:02.784 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-16T18:59:02.833 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-16T18:59:02.871 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 49/172 2024-08-16T18:59:02.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-16T18:59:03.057 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-16T18:59:03.116 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-16T18:59:03.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-16T18:59:03.243 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-16T18:59:03.275 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-16T18:59:03.311 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-16T18:59:03.357 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-16T18:59:03.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-16T18:59:03.436 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-16T18:59:03.479 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-16T18:59:03.617 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-16T18:59:03.722 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-16T18:59:03.932 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-16T18:59:04.036 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-16T18:59:04.072 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 65/172 2024-08-16T18:59:04.112 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-16T18:59:04.138 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-16T18:59:04.162 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-16T18:59:04.236 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-16T18:59:04.363 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-16T18:59:04.411 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-16T18:59:04.438 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-16T18:59:04.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-16T18:59:04.512 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-16T18:59:04.543 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-16T18:59:04.573 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-16T18:59:04.608 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-16T18:59:04.641 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-16T18:59:04.683 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-16T18:59:04.717 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-16T18:59:04.804 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-16T18:59:04.847 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-16T18:59:04.878 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-16T18:59:04.981 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-16T18:59:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:05.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-16T18:59:05.810 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-16T18:59:05.812 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-08-16T18:59:05.843 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-16T18:59:05.878 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-16T18:59:05.916 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-16T18:59:05.942 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-16T18:59:05.970 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-16T18:59:05.998 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-16T18:59:06.011 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-16T18:59:06.036 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-16T18:59:06.043 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-16T18:59:06.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-16T18:59:06.130 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-16T18:59:06.171 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-16T18:59:06.350 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-16T18:59:06.377 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-16T18:59:06.446 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-16T18:59:06.455 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-16T18:59:06.554 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:06.680 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:06.830 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:06.927 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:06.959 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:59:07.011 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-16T18:59:07.099 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-16T18:59:07.204 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-16T18:59:07.217 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-16T18:59:07.258 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-16T18:59:07.284 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:07.374 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:07.404 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:59:07.467 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:59:07.488 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:59:07.512 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:59:07.717 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:07.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-16T18:59:08.122 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-08-16T18:59:08.216 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-16T18:59:08.222 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:08.264 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-16T18:59:08.282 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:08.317 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-16T18:59:08.351 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-16T18:59:08.376 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-16T18:59:08.378 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:08.419 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-16T18:59:08.466 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-16T18:59:08.467 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:08.555 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-16T18:59:08.583 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-16T18:59:08.616 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-16T18:59:08.683 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:08.792 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-16T18:59:08.803 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-16T18:59:08.868 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:08.888 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-16T18:59:08.922 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-16T18:59:08.925 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-16T18:59:08.957 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-16T18:59:08.971 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:08.991 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:09.020 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:09.033 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-16T18:59:09.066 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:09.084 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-16T18:59:09.093 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 105/172 2024-08-16T18:59:09.113 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:09.124 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-16T18:59:09.128 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-16T18:59:09.140 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:59:09.173 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-16T18:59:09.284 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-16T18:59:09.317 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-16T18:59:09.344 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-16T18:59:09.357 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-16T18:59:09.413 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-16T18:59:09.416 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-16T18:59:09.441 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:09.484 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-16T18:59:09.514 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-16T18:59:09.546 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:09.558 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-16T18:59:09.576 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:59:09.613 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-16T18:59:09.636 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-16T18:59:09.638 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 15/172 2024-08-16T18:59:09.641 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:09.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:59:09.667 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 36/172 2024-08-16T18:59:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-16T18:59:09.697 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:09.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 37/172 2024-08-16T18:59:09.726 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 109/172 2024-08-16T18:59:09.731 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 38/172 2024-08-16T18:59:09.805 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-16T18:59:09.855 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-16T18:59:09.877 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:09.933 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-16T18:59:09.970 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-16T18:59:10.138 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-16T18:59:10.195 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-16T18:59:10.250 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-16T18:59:10.309 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:10.370 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-16T18:59:10.456 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:10.538 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:10.729 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-16T18:59:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-16T18:59:10.835 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-16T18:59:10.889 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-16T18:59:10.922 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-16T18:59:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-16T18:59:10.960 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-16T18:59:10.968 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-16T18:59:11.003 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-16T18:59:11.009 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-16T18:59:11.047 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 49/172 2024-08-16T18:59:11.068 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-16T18:59:11.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-16T18:59:11.113 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-16T18:59:11.153 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-16T18:59:11.197 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-16T18:59:11.229 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-16T18:59:11.257 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-16T18:59:11.266 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-16T18:59:11.320 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-16T18:59:11.333 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-16T18:59:11.380 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-16T18:59:11.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-16T18:59:11.454 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-16T18:59:11.531 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-16T18:59:11.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 36/172 2024-08-16T18:59:11.575 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-16T18:59:11.619 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 37/172 2024-08-16T18:59:11.645 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 38/172 2024-08-16T18:59:11.715 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-16T18:59:11.716 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-16T18:59:11.752 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-16T18:59:11.760 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-16T18:59:11.788 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-16T18:59:11.835 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-16T18:59:11.836 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-16T18:59:11.870 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-16T18:59:11.914 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-16T18:59:11.956 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-16T18:59:11.972 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-16T18:59:11.996 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:12.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-16T18:59:12.031 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:12.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-16T18:59:12.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-16T18:59:12.191 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-16T18:59:12.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-16T18:59:12.332 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 65/172 2024-08-16T18:59:12.373 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-16T18:59:12.406 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-16T18:59:12.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-16T18:59:12.515 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-16T18:59:12.567 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-16T18:59:12.577 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:12.617 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:12.644 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-16T18:59:12.645 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-16T18:59:12.672 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-16T18:59:12.679 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-16T18:59:12.707 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:12.707 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-16T18:59:12.707 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-16T18:59:12.708 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:12.708 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-16T18:59:12.708 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-16T18:59:12.735 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:12.745 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-16T18:59:12.757 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-16T18:59:12.781 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-16T18:59:12.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 49/172 2024-08-16T18:59:12.821 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-16T18:59:12.846 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-16T18:59:12.851 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-16T18:59:12.896 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-16T18:59:12.927 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-16T18:59:12.970 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-16T18:59:13.002 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-16T18:59:13.003 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-16T18:59:13.048 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-16T18:59:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-16T18:59:13.103 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-16T18:59:13.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-16T18:59:13.174 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-16T18:59:13.192 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-16T18:59:13.225 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-16T18:59:13.260 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-16T18:59:13.281 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-16T18:59:13.307 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-16T18:59:13.342 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:13.351 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-16T18:59:13.395 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-16T18:59:13.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-16T18:59:13.574 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-16T18:59:13.621 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-16T18:59:13.655 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-16T18:59:13.766 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-16T18:59:13.804 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 65/172 2024-08-16T18:59:13.844 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-16T18:59:13.870 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-16T18:59:13.895 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-16T18:59:13.969 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-16T18:59:14.098 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-16T18:59:14.108 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-16T18:59:14.135 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-16T18:59:14.155 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-16T18:59:14.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-16T18:59:14.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-16T18:59:14.293 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-16T18:59:14.331 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-16T18:59:14.345 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-16T18:59:14.370 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-16T18:59:14.391 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-16T18:59:14.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-16T18:59:14.440 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-16T18:59:14.474 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-16T18:59:14.501 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-16T18:59:14.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-16T18:59:14.549 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-16T18:59:14.584 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-16T18:59:14.672 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-16T18:59:14.714 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-16T18:59:14.744 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-16T18:59:14.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-16T18:59:14.848 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-16T18:59:14.896 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:15.012 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-16T18:59:15.039 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-16T18:59:15.126 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-16T18:59:15.628 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-16T18:59:15.670 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-16T18:59:15.701 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-16T18:59:15.799 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-16T18:59:15.829 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-16T18:59:15.856 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-16T18:59:15.901 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-16T18:59:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-16T18:59:15.990 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-16T18:59:16.037 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-16T18:59:16.228 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-16T18:59:16.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-16T18:59:16.330 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-16T18:59:16.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-16T18:59:17.099 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-16T18:59:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-16T18:59:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-16T18:59:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-16T18:59:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-16T18:59:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-16T18:59:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 105/172 2024-08-16T18:59:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-16T18:59:18.350 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-16T18:59:18.448 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-16T18:59:18.454 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-16T18:59:18.476 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-16T18:59:18.479 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:18.501 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-16T18:59:18.552 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:18.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 109/172 2024-08-16T18:59:18.812 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-16T18:59:18.856 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:18.951 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 104/172 2024-08-16T18:59:18.978 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 105/172 2024-08-16T18:59:19.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-16T18:59:19.489 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-16T18:59:19.526 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:19.598 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 108/172 2024-08-16T18:59:19.628 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 109/172 2024-08-16T18:59:21.059 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:21.094 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:21.676 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:21.723 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:21.819 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:21.819 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-16T18:59:21.819 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-16T18:59:21.820 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:21.853 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:21.949 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:22.150 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:22.699 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 110/172 2024-08-16T18:59:22.737 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:22.831 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 111/172 2024-08-16T18:59:22.831 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-16T18:59:22.831 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-16T18:59:22.831 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:22.863 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-08-16T18:59:24.295 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-08-16T18:59:24.296 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-08-16T18:59:24.296 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:24.473 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:24.490 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:24.490 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:24.490 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-16T18:59:24.490 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:25.960 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:25.986 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:25.986 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:25.986 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-16T18:59:25.986 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:25.986 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:25.987 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:26.210 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:26.233 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:26.234 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:26.234 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-16T18:59:26.234 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:26.234 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:26.234 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:26.670 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:26.693 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:26.803 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-16T18:59:26.857 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-16T18:59:26.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-16T18:59:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-16T18:59:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-16T18:59:27.224 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-16T18:59:27.320 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-16T18:59:27.469 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-16T18:59:27.511 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-16T18:59:27.543 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-16T18:59:27.579 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-16T18:59:28.014 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:28.043 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:28.078 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 129/172 2024-08-16T18:59:28.141 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 130/172 2024-08-16T18:59:28.179 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:29.221 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:29.260 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:30.225 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:30.248 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:30.350 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:30.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 134/172 2024-08-16T18:59:30.550 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:30.574 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:30.574 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:30.574 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-16T18:59:30.574 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:30.574 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:30.575 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:30.635 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:30.646 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:30.743 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 137/172 2024-08-16T18:59:30.791 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 138/172 2024-08-16T18:59:31.704 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-16T18:59:31.730 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:31.753 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:31.754 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:31.754 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-16T18:59:31.754 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:31.754 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:31.754 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:31.797 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:31.819 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:31.820 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:31.820 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-16T18:59:31.820 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:31.864 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:31.888 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:32.190 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:32.213 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-08-16T18:59:33.822 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-08-16T18:59:33.823 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:34.003 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:34.021 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:34.021 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:34.022 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-16T18:59:34.022 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 112/172 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-08-16T18:59:34.382 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:34.555 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:34.572 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 113/172 2024-08-16T18:59:34.572 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:34.572 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-16T18:59:34.572 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:35.573 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:35.601 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:35.602 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:35.602 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-16T18:59:35.602 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:35.602 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:35.602 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:35.840 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:35.865 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:35.865 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:35.865 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-16T18:59:35.865 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:35.865 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:35.866 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:36.059 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 114/172 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-16T18:59:36.085 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:36.318 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:36.333 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 115/172 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-16T18:59:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:36.357 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:36.358 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:36.358 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-16T18:59:36.358 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:36.358 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:36.358 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:36.480 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-16T18:59:36.539 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-16T18:59:36.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-16T18:59:36.753 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 144/172 2024-08-16T18:59:36.778 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:36.794 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 145/172 2024-08-16T18:59:36.798 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 116/172 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-16T18:59:36.800 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:36.835 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 146/172 2024-08-16T18:59:36.859 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-16T18:59:36.867 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-16T18:59:36.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-16T18:59:36.899 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-16T18:59:36.940 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-16T18:59:36.956 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-16T18:59:37.020 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-16T18:59:37.038 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-16T18:59:37.082 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:37.124 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-16T18:59:37.157 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 151/172 2024-08-16T18:59:37.182 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-16T18:59:37.198 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-16T18:59:37.208 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-16T18:59:37.226 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-16T18:59:37.227 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:37.257 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-16T18:59:37.261 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-16T18:59:37.282 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-16T18:59:37.284 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-16T18:59:37.340 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-16T18:59:37.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-16T18:59:37.577 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-16T18:59:37.611 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-16T18:59:37.643 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-16T18:59:37.680 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-16T18:59:37.748 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:37.775 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:37.807 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 129/172 2024-08-16T18:59:37.871 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 130/172 2024-08-16T18:59:37.887 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:37.906 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:37.942 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:38.107 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:38.137 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 128/172 2024-08-16T18:59:38.169 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 129/172 2024-08-16T18:59:38.231 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 130/172 2024-08-16T18:59:38.260 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:38.790 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 156/172 2024-08-16T18:59:38.876 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-16T18:59:38.916 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-16T18:59:39.008 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-16T18:59:39.018 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:39.046 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-16T18:59:39.049 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:39.071 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-16T18:59:39.097 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:39.123 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-16T18:59:39.123 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:39.224 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-16T18:59:39.257 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:39.323 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 131/172 2024-08-16T18:59:39.341 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:39.366 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-16T18:59:39.371 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:39.372 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:39.372 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:39.569 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:39.652 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:39.678 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:39.704 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-16T18:59:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:40.074 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:40.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:40.208 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:40.340 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 132/172 2024-08-16T18:59:40.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 134/172 2024-08-16T18:59:40.383 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:40.419 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:40.444 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:40.486 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 133/172 2024-08-16T18:59:40.494 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:40.506 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:40.616 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 137/172 2024-08-16T18:59:40.655 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 134/172 2024-08-16T18:59:40.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 138/172 2024-08-16T18:59:40.678 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:40.701 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 135/172 2024-08-16T18:59:40.702 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:40.702 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-16T18:59:40.702 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:40.702 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-16T18:59:40.702 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:40.746 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:40.758 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 136/172 2024-08-16T18:59:40.857 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 137/172 2024-08-16T18:59:40.902 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 138/172 2024-08-16T18:59:41.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-16T18:59:41.680 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:41.705 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:41.705 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:41.706 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-16T18:59:41.706 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:41.706 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:41.706 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:41.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:41.780 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:41.780 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-16T18:59:41.780 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:41.818 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-16T18:59:41.831 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:41.848 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:41.856 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 140/172 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-16T18:59:41.872 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:41.917 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:41.938 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1363.gd863d 141/172 2024-08-16T18:59:41.939 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:41.939 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-16T18:59:41.939 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:41.984 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 142/172 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-16T18:59:42.007 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:42.182 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:42.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:42.309 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:42.332 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 143/172 2024-08-16T18:59:42.332 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-16T18:59:42.333 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-16T18:59:42.333 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:42.333 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-16T18:59:42.333 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 1/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 3/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 4/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 5/172 2024-08-16T18:59:45.411 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 6/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 7/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 11/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 12/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 13/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 15/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 16/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 17/172 2024-08-16T18:59:45.412 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 18/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 19/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 20/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 21/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 22/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 23/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 24/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 25/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 26/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 27/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 28/172 2024-08-16T18:59:45.413 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 29/172 2024-08-16T18:59:45.425 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 30/172 2024-08-16T18:59:45.425 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 31/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 32/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 33/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 34/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 35/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 36/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 37/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 38/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 39/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-16T18:59:45.426 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-16T18:59:45.427 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-16T18:59:45.428 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-16T18:59:45.429 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-16T18:59:45.430 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-16T18:59:45.431 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-16T18:59:45.432 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-16T18:59:45.433 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-16T18:59:45.434 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 151/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:45.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-16T18:59:45.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-16T18:59:45.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-16T18:59:45.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-16T18:59:45.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:45.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-16T18:59:46.865 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 144/172 2024-08-16T18:59:46.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 145/172 2024-08-16T18:59:46.946 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 146/172 2024-08-16T18:59:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-16T18:59:46.998 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-16T18:59:47.028 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 144/172 2024-08-16T18:59:47.069 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 145/172 2024-08-16T18:59:47.120 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 146/172 2024-08-16T18:59:47.150 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-16T18:59:47.178 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:47.180 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-16T18:59:47.216 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-16T18:59:47.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 151/172 2024-08-16T18:59:47.290 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-16T18:59:47.319 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:47.365 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-16T18:59:47.382 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:47.431 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-16T18:59:47.464 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 151/172 2024-08-16T18:59:47.504 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-16T18:59:47.534 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:47.580 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-16T18:59:47.946 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:47.981 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:48.001 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:48.191 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:48.235 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-16T18:59:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-16T18:59:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-16T18:59:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.14-1.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-16T18:59:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-16T18:59:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-16T18:59:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-16T18:59:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-16T18:59:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-16T18:59:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-16T18:59:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-16T18:59:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-16T18:59:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-16T18:59:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T18:59:48.257 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-08-16T18:59:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-16T18:59:48.664 DEBUG:teuthology.parallel:result is None 2024-08-16T18:59:48.846 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 156/172 2024-08-16T18:59:48.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-16T18:59:48.967 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-16T18:59:49.056 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-16T18:59:49.108 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-16T18:59:49.133 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-16T18:59:49.158 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:49.161 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 156/172 2024-08-16T18:59:49.184 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-16T18:59:49.184 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:49.269 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:49.271 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-16T18:59:49.302 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-16T18:59:49.302 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:49.305 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-16T18:59:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:49.412 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-16T18:59:49.417 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.417 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.426 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-16T18:59:49.467 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-16T18:59:49.492 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-16T18:59:49.517 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:49.543 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-16T18:59:49.544 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:49.597 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.598 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:49.646 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-16T18:59:49.674 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:49.680 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-16T18:59:49.680 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:49.698 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:49.724 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-16T18:59:49.764 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-16T18:59:49.789 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-16T18:59:49.795 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.795 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.797 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:49.983 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-16T18:59:49.983 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:50.059 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-16T18:59:50.086 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:50.118 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-16T18:59:50.194 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 1/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 3/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 4/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 5/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 6/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 7/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:55.428 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 11/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 12/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 13/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 15/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 16/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 17/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 18/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 19/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 20/172 2024-08-16T18:59:55.429 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 21/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 22/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 23/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 24/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 25/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 26/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 27/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 28/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 29/172 2024-08-16T18:59:55.430 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 30/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 31/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 32/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 33/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 34/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 35/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 36/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 37/172 2024-08-16T18:59:55.445 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 38/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 39/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-16T18:59:55.446 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-16T18:59:55.447 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-16T18:59:55.448 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:55.449 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-16T18:59:55.450 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-16T18:59:55.451 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-16T18:59:55.452 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-16T18:59:55.453 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-16T18:59:55.454 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 151/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:55.455 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:55.456 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-16T18:59:55.815 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 1/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 3/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 4/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1363.gd863d 5/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 6/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 7/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 8/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 9/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 10/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 11/172 2024-08-16T18:59:55.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 12/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 13/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 14/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_ 15/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 16/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 17/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_6 18/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86 19/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 20/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el 21/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.1.0-1363.gd863d21c.el9. 22/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_6 23/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 24/172 2024-08-16T18:59:55.817 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 25/172 2024-08-16T18:59:55.818 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 26/172 2024-08-16T18:59:55.818 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 27/172 2024-08-16T18:59:55.818 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 28/172 2024-08-16T18:59:55.818 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 29/172 2024-08-16T18:59:55.818 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c. 30/172 2024-08-16T18:59:55.832 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noa 31/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.n 32/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1363.gd86 33/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el 34/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 35/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.e 36/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 37/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 38/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 39/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-16T18:59:55.833 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-16T18:59:55.834 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-16T18:59:55.835 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-16T18:59:55.836 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-16T18:59:55.837 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-16T18:59:55.838 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-16T18:59:55.839 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-16T18:59:55.840 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-16T18:59:55.841 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 149/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 151/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-16T18:59:55.842 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-16T18:59:55.843 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-16T18:59:55.844 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.307 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2024-08-16T18:59:58.308 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.310 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.311 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-16T18:59:58.312 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.14-1.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-16T18:59:58.313 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-16T18:59:58.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-16T18:59:58.315 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-16T18:59:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-16T18:59:58.317 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-16T18:59:58.318 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.319 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T18:59:58.320 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-08-16T18:59:58.745 DEBUG:teuthology.parallel:result is None 2024-08-16T18:59:58.782 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-16T18:59:58.782 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:58.782 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.783 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.784 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.785 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.1.0-1363.gd863d21c.el9.noarch 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-16T18:59:58.786 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-16T18:59:58.787 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-16T18:59:58.788 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.789 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-16T18:59:58.790 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-16T18:59:58.791 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-16T18:59:58.792 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-16T18:59:58.793 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.1.0-1363.gd863d21c.el9.x86_64 2024-08-16T18:59:58.794 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T18:59:58.795 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-08-16T18:59:59.211 DEBUG:teuthology.parallel:result is None 2024-08-16T18:59:59.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:59:59.354 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-16T18:59:59.377 INFO:teuthology.orchestra.run.smithi062.stdout:19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.378 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.378 INFO:teuthology.task.install:The correct ceph version 19.1.0-1363.gd863d21c is installed. 2024-08-16T18:59:59.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:59:59.514 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-16T18:59:59.539 INFO:teuthology.orchestra.run.smithi155.stdout:19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.539 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.540 INFO:teuthology.task.install:The correct ceph version 19.1.0-1363.gd863d21c is installed. 2024-08-16T18:59:59.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T18:59:59.679 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-16T18:59:59.705 INFO:teuthology.orchestra.run.smithi161.stdout:19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.705 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1363.gd863d21c.el9 2024-08-16T18:59:59.705 INFO:teuthology.task.install:The correct ceph version 19.1.0-1363.gd863d21c is installed. 2024-08-16T18:59:59.707 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-16T18:59:59.707 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T18:59:59.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-16T18:59:59.736 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T18:59:59.736 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-16T18:59:59.768 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T18:59:59.768 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-16T18:59:59.798 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-16T18:59:59.798 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T18:59:59.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-08-16T18:59:59.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-16T18:59:59.892 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T18:59:59.892 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-08-16T18:59:59.921 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-16T18:59:59.987 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T18:59:59.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-08-16T19:00:00.015 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-16T19:00:00.083 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-16T19:00:00.083 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:00:00.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-16T19:00:00.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-16T19:00:00.178 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:00:00.179 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-16T19:00:00.207 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-16T19:00:00.274 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:00:00.275 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-16T19:00:00.300 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-16T19:00:00.364 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-16T19:00:00.364 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:00:00.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-08-16T19:00:00.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-16T19:00:00.457 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:00:00.457 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-08-16T19:00:00.484 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-16T19:00:00.549 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:00:00.549 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-08-16T19:00:00.574 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-16T19:00:00.639 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-16T19:00:00.739 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-16T19:00:00.739 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T19:00:00.740 INFO:tasks.cephadm:Cluster fsid is bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:00:00.740 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-16T19:00:00.740 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.62', 'mon.b': '172.21.15.155', 'mon.c': '172.21.15.161'} 2024-08-16T19:00:00.740 INFO:tasks.cephadm:First mon is mon.a on smithi062 2024-08-16T19:00:00.740 INFO:tasks.cephadm:First mgr is x 2024-08-16T19:00:00.740 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-16T19:00:00.740 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-08-16T19:00:00.769 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-08-16T19:00:00.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-08-16T19:00:00.822 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-16T19:00:00.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T19:00:00.961 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd863d21c9d1db9c3453c09dacd68766969720a44', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55836/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1363-gd863d21c', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1363.gd863d21c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-14 17:19:06.671644', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-16T19:00:01.101 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 d863d21c9d1db9c3453c09dacd68766969720a44 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T19:00:01.103 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/x86_64/flavors/default/cephadm 2024-08-16T19:00:01.103 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/x86_64/flavors/default/cephadm 2024-08-16T19:00:01.103 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:01.743 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 784594 Aug 16 19:00 /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:01.744 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:02.449 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 784594 Aug 16 19:00 /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:02.450 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/d863d21c9d1db9c3453c09dacd68766969720a44/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:03.160 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 784594 Aug 16 19:00 /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:03.160 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:03.180 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:03.199 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-16T19:00:03.234 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 on all hosts... 2024-08-16T19:00:03.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 pull 2024-08-16T19:00:03.237 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 pull 2024-08-16T19:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 pull 2024-08-16T19:00:04.102 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44... 2024-08-16T19:00:04.189 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44... 2024-08-16T19:00:04.211 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44... 2024-08-16T19:00:31.620 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc)", 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147", 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb" 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-08-16T19:00:31.621 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc)", 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147", 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb" 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:00:49.658 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc)", 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147", 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb" 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-08-16T19:00:53.646 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-16T19:00:53.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-08-16T19:00:53.707 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-08-16T19:00:53.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-08-16T19:00:53.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-08-16T19:00:53.795 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-08-16T19:00:53.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-08-16T19:00:53.847 INFO:tasks.cephadm:Writing seed config... 2024-08-16T19:00:53.848 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-16T19:00:53.848 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-16T19:00:53.849 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-16T19:00:53.850 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-16T19:00:53.851 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-16T19:00:53.852 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-16T19:00:53.853 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:00:53.853 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-16T19:00:53.871 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bd001ebc-5c01-11ef-bccf-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-16T19:00:53.872 DEBUG:teuthology.orchestra.run.smithi062:mon.a> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service 2024-08-16T19:00:53.914 DEBUG:teuthology.orchestra.run.smithi062:mgr.x> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service 2024-08-16T19:00:53.957 INFO:tasks.cephadm:Bootstrapping... 2024-08-16T19:00:53.958 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 -v bootstrap --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.62 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:00:54.174 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-08-16T19:00:54.174 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44', '-v', 'bootstrap', '--fsid', 'bd001ebc-5c01-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.62', '--skip-admin-label'] 2024-08-16T19:00:54.175 INFO:teuthology.orchestra.run.smithi062.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-16T19:00:54.175 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying podman|docker is present... 2024-08-16T19:00:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.1.2 2024-08-16T19:00:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying lvm2 is present... 2024-08-16T19:00:54.201 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying time synchronization is in place... 2024-08-16T19:00:54.209 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-16T19:00:54.209 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-16T19:00:54.215 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-16T19:00:54.215 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:00:54.222 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-08-16T19:00:54.228 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-08-16T19:00:54.228 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-08-16T19:00:54.229 INFO:teuthology.orchestra.run.smithi062.stdout:Repeating the final host check... 2024-08-16T19:00:54.253 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout 5.1.2 2024-08-16T19:00:54.253 INFO:teuthology.orchestra.run.smithi062.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-16T19:00:54.253 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl is present 2024-08-16T19:00:54.253 INFO:teuthology.orchestra.run.smithi062.stdout:lvcreate is present 2024-08-16T19:00:54.260 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-16T19:00:54.260 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-16T19:00:54.266 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-16T19:00:54.266 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:00:54.273 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout enabled 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout active 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:Unit chronyd.service is enabled and running 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:Host looks OK 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:Cluster fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:Acquiring lock 140338273781888 on /run/cephadm/bd001ebc-5c01-11ef-bccf-c7b262605968.lock 2024-08-16T19:00:54.280 INFO:teuthology.orchestra.run.smithi062.stdout:Lock 140338273781888 acquired on /run/cephadm/bd001ebc-5c01-11ef-bccf-c7b262605968.lock 2024-08-16T19:00:54.281 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 3300 ... 2024-08-16T19:00:54.281 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying IP 172.21.15.62 port 6789 ... 2024-08-16T19:00:54.281 INFO:teuthology.orchestra.run.smithi062.stdout:Base mon IP(s) is [172.21.15.62:3300, 172.21.15.62:6789], mon addrv is [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-08-16T19:00:54.285 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.62 metric 100 2024-08-16T19:00:54.285 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.62 metric 100 2024-08-16T19:00:54.287 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-16T19:00:54.287 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-08-16T19:00:54.288 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1772sec hoplimit 64 pref medium 2024-08-16T19:00:54.290 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-16T19:00:54.290 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-16T19:00:54.290 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-16T19:00:54.290 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-16T19:00:54.290 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-08-16T19:00:54.291 INFO:teuthology.orchestra.run.smithi062.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-16T19:00:54.291 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-08-16T19:00:54.291 INFO:teuthology.orchestra.run.smithi062.stdout:Mon IP `172.21.15.62` is in CIDR network `172.21.0.0/20` 2024-08-16T19:00:54.291 INFO:teuthology.orchestra.run.smithi062.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-16T19:00:54.292 INFO:teuthology.orchestra.run.smithi062.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-16T19:00:54.293 INFO:teuthology.orchestra.run.smithi062.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44... 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stdout ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44... 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Getting image source signatures 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:eca91c7dce8cb7c87072c6ad3efbc79b0f9c3a123a433c450b8b9e990c9baa18 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Copying config sha256:ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 2024-08-16T19:00:54.761 INFO:teuthology.orchestra.run.smithi062.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-16T19:00:55.690 INFO:teuthology.orchestra.run.smithi062.stdout:ceph: stdout ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc) 2024-08-16T19:00:55.690 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph version: ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc) 2024-08-16T19:00:55.690 INFO:teuthology.orchestra.run.smithi062.stdout:Extracting ceph user uid/gid from container image... 2024-08-16T19:00:56.472 INFO:teuthology.orchestra.run.smithi062.stdout:stat: stdout 167 167 2024-08-16T19:00:56.472 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial keys... 2024-08-16T19:00:57.530 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDoob9m4vF3LhAA22+YBd8cGxlas1/ClLp9AQ== 2024-08-16T19:00:58.507 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDpob9mKJnGNxAATNR/VBCgAxxpCdNBwtDtUg== 2024-08-16T19:00:59.497 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph-authtool: stdout AQDqob9mOrCmMRAA7A1hN66p8Z8kBgSZsHAQkA== 2024-08-16T19:00:59.497 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial monmap... 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool for a [v2:172.21.15.62:3300,v1:172.21.15.62:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: set fsid to bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-16T19:01:00.278 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:01:00.279 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mon... 2024-08-16T19:01:01.192 INFO:teuthology.orchestra.run.smithi062.stdout:create mon.a on 2024-08-16T19:01:01.413 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-16T19:01:01.580 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-16T19:01:01.770 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target. 2024-08-16T19:01:01.770 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target. 2024-08-16T19:01:02.098 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a 2024-08-16T19:01:02.098 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-16T19:01:02.277 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target.wants/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@.service. 2024-08-16T19:01:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-16T19:01:02.833 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-08-16T19:01:02.840 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-16T19:01:02.840 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:01:02.840 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-08-16T19:01:02.840 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-08-16T19:01:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon to start... 2024-08-16T19:01:02.841 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mon... 2024-08-16T19:01:02.966 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:02 smithi062 systemd[1]: Started Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: monmap epoch 1 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: last_changed 2024-08-16T19:00:59.803011+0000 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: min_mon_release 19 (squid) 2024-08-16T19:01:04.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: election_strategy: 1 2024-08-16T19:01:04.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:01:04.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: fsmap 2024-08-16T19:01:04.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: osdmap e1: 0 total, 0 up, 0 in 2024-08-16T19:01:04.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:03 smithi062 ceph-mon[32624]: mgrmap e1: no daemons active 2024-08-16T19:01:05.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:05 smithi062 ceph-mon[32624]: from='client.? 172.21.15.62:0/1940383048' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-16T19:01:11.847 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout cluster: 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout id: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout services: 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 2s) 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-16T19:01:11.854 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout data: 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout pgs: 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f16667c6640 1 Processor -- start 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f16667c6640 1 -- start start 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f16667c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f16667c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f166007e0a0 con 0x7f166007dca0 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f165ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.296+0000 7f165ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50780/0 (socket says 172.21.15.62:50780) 2024-08-16T19:01:11.855 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.297+0000 7f165ffff640 1 -- 172.21.15.62:0/725029501 learned_addr learned my addr 172.21.15.62:0/725029501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.297+0000 7f165ffff640 1 -- 172.21.15.62:0/725029501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166007c7a0 con 0x7f166007dca0 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f165ffff640 1 --2- 172.21.15.62:0/725029501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1650009820 tx=0x7f165002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92387ee7ad4786f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f165effd640 1 -- 172.21.15.62:0/725029501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1650037070 con 0x7f166007dca0 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f165effd640 1 -- 172.21.15.62:0/725029501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1650032440 con 0x7f166007dca0 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f165effd640 1 -- 172.21.15.62:0/725029501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1650032740 con 0x7f166007dca0 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 -- 172.21.15.62:0/725029501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 msgr2=0x7f166007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 --2- 172.21.15.62:0/725029501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1650009820 tx=0x7f165002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 -- 172.21.15.62:0/725029501 shutdown_connections 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 --2- 172.21.15.62:0/725029501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166007c260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 -- 172.21.15.62:0/725029501 >> 172.21.15.62:0/725029501 conn(0x7f1660102c50 msgr2=0x7f1660105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 -- 172.21.15.62:0/725029501 shutdown_connections 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.299+0000 7f16667c6640 1 -- 172.21.15.62:0/725029501 wait complete. 2024-08-16T19:01:11.856 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.300+0000 7f16667c6640 1 Processor -- start 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.300+0000 7f16667c6640 1 -- start start 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f16667c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f16667c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f166007cce0 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50794/0 (socket says 172.21.15.62:50794) 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165ffff640 1 -- 172.21.15.62:0/1940383048 learned_addr learned my addr 172.21.15.62:0/1940383048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165ffff640 1 -- 172.21.15.62:0/1940383048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166019d230 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165ffff640 1 --2- 172.21.15.62:0/1940383048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f16500090b0 tx=0x7f16500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1650041070 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165003c7a0 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.301+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1650037040 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.302+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166019d4c0 con 0x7f166007dca0 2024-08-16T19:01:11.857 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.302+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601a01a0 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.302+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1650044070 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.302+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f165003d3f0 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.303+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166007c2e0 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.305+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f1650046450 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.367+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f166002d810 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.368+0000 7f165d7fa640 1 -- 172.21.15.62:0/1940383048 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f1650046630 con 0x7f166007dca0 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.368+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 msgr2=0x7f166019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.368+0000 7f16667c6640 1 --2- 172.21.15.62:0/1940383048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f16500090b0 tx=0x7f16500046c0 comp rx=0 tx=0).stop 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.369+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 shutdown_connections 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.369+0000 7f16667c6640 1 --2- 172.21.15.62:0/1940383048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f166007dca0 0x7f166019ccf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.369+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 >> 172.21.15.62:0/1940383048 conn(0x7f1660102c50 msgr2=0x7f1660103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:11.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.369+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 shutdown_connections 2024-08-16T19:01:11.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:05.369+0000 7f16667c6640 1 -- 172.21.15.62:0/1940383048 wait complete. 2024-08-16T19:01:11.859 INFO:teuthology.orchestra.run.smithi062.stdout:mon is available 2024-08-16T19:01:11.859 INFO:teuthology.orchestra.run.smithi062.stdout:Assimilating anything we can from ceph.conf... 2024-08-16T19:01:13.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:12 smithi062 ceph-mon[32624]: from='client.? 172.21.15.62:0/2570718500' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-16T19:01:13.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:12 smithi062 ceph-mon[32624]: from='client.? 172.21.15.62:0/2570718500' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-16T19:01:13.358 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:13.358 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-16T19:01:13.359 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-08-16T19:01:13.360 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.683+0000 7ff1028f8640 1 Processor -- start 2024-08-16T19:01:13.361 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff1028f8640 1 -- start start 2024-08-16T19:01:13.362 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff1028f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:13.362 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff1028f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc07be30 con 0x7ff0fc107d60 2024-08-16T19:01:13.362 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:13.362 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36250/0 (socket says 172.21.15.62:36250) 2024-08-16T19:01:13.362 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff0fbfff640 1 -- 172.21.15.62:0/3397667354 learned_addr learned my addr 172.21.15.62:0/3397667354 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.684+0000 7ff0fbfff640 1 -- 172.21.15.62:0/3397667354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc10a690 con 0x7ff0fc107d60 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.685+0000 7ff0fbfff640 1 --2- 172.21.15.62:0/3397667354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009820 tx=0x7ff0ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be2c8f89c66e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.685+0000 7ff0faffd640 1 -- 172.21.15.62:0/3397667354 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec037070 con 0x7ff0fc107d60 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.685+0000 7ff0faffd640 1 -- 172.21.15.62:0/3397667354 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec032440 con 0x7ff0fc107d60 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.685+0000 7ff0faffd640 1 -- 172.21.15.62:0/3397667354 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec032740 con 0x7ff0fc107d60 2024-08-16T19:01:13.363 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 -- 172.21.15.62:0/3397667354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 msgr2=0x7ff0fc10a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 --2- 172.21.15.62:0/3397667354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009820 tx=0x7ff0ec02a270 comp rx=0 tx=0).stop 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 -- 172.21.15.62:0/3397667354 shutdown_connections 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 --2- 172.21.15.62:0/3397667354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc10a150 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 -- 172.21.15.62:0/3397667354 >> 172.21.15.62:0/3397667354 conn(0x7ff0fc101a00 msgr2=0x7ff0fc103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 -- 172.21.15.62:0/3397667354 shutdown_connections 2024-08-16T19:01:13.364 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.686+0000 7ff1028f8640 1 -- 172.21.15.62:0/3397667354 wait complete. 2024-08-16T19:01:13.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff1028f8640 1 Processor -- start 2024-08-16T19:01:13.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff1028f8640 1 -- start start 2024-08-16T19:01:13.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff1028f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:13.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff1028f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc10abd0 con 0x7ff0fc107d60 2024-08-16T19:01:13.365 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36264/0 (socket says 172.21.15.62:36264) 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff0fbfff640 1 -- 172.21.15.62:0/2570718500 learned_addr learned my addr 172.21.15.62:0/2570718500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.687+0000 7ff0fbfff640 1 -- 172.21.15.62:0/2570718500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc19d260 con 0x7ff0fc107d60 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0fbfff640 1 --2- 172.21.15.62:0/2570718500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0090b0 tx=0x7ff0ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec041070 con 0x7ff0fc107d60 2024-08-16T19:01:13.366 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec03c7a0 con 0x7ff0fc107d60 2024-08-16T19:01:13.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc19d4f0 con 0x7ff0fc107d60 2024-08-16T19:01:13.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec037040 con 0x7ff0fc107d60 2024-08-16T19:01:13.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1a01d0 con 0x7ff0fc107d60 2024-08-16T19:01:13.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff0ec044070 con 0x7ff0fc107d60 2024-08-16T19:01:13.367 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.688+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff0ec03d3f0 con 0x7ff0fc107d60 2024-08-16T19:01:13.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.689+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc1081c0 con 0x7ff0fc107d60 2024-08-16T19:01:13.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.691+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7ff0ec046450 con 0x7ff0fc107d60 2024-08-16T19:01:13.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.771+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff0fc02d7a0 con 0x7ff0fc107d60 2024-08-16T19:01:13.368 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.775+0000 7ff0f97fa640 1 -- 172.21.15.62:0/2570718500 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7ff0ec046630 con 0x7ff0fc107d60 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 msgr2=0x7ff0fc19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 --2- 172.21.15.62:0/2570718500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0090b0 tx=0x7ff0ec0046c0 comp rx=0 tx=0).stop 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 shutdown_connections 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 --2- 172.21.15.62:0/2570718500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0fc107d60 0x7ff0fc19cd20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 >> 172.21.15.62:0/2570718500 conn(0x7ff0fc101a00 msgr2=0x7ff0fc101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 shutdown_connections 2024-08-16T19:01:13.369 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:12.776+0000 7ff1028f8640 1 -- 172.21.15.62:0/2570718500 wait complete. 2024-08-16T19:01:13.370 INFO:teuthology.orchestra.run.smithi062.stdout:Generating new minimal ceph.conf... 2024-08-16T19:01:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:13 smithi062 ceph-mon[32624]: from='client.? 172.21.15.62:0/871613717' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.757+0000 7f446f04f640 1 Processor -- start 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446f04f640 1 -- start start 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446f04f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446f04f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f446807bd90 con 0x7f4468105bf0 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446cdeb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446cdeb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36274/0 (socket says 172.21.15.62:36274) 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446cdeb640 1 -- 172.21.15.62:0/169726492 learned_addr learned my addr 172.21.15.62:0/169726492 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.758+0000 7f446cdeb640 1 -- 172.21.15.62:0/169726492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468106530 con 0x7f4468105bf0 2024-08-16T19:01:14.337 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.759+0000 7f446cdeb640 1 --2- 172.21.15.62:0/169726492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4450009820 tx=0x7f445002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395bca628d19a61b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.759+0000 7f44677fe640 1 -- 172.21.15.62:0/169726492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4450037070 con 0x7f4468105bf0 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.759+0000 7f44677fe640 1 -- 172.21.15.62:0/169726492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4450032900 con 0x7f4468105bf0 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.759+0000 7f44677fe640 1 -- 172.21.15.62:0/169726492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4450030850 con 0x7f4468105bf0 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 -- 172.21.15.62:0/169726492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 msgr2=0x7f4468105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 --2- 172.21.15.62:0/169726492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4450009820 tx=0x7f445002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 -- 172.21.15.62:0/169726492 shutdown_connections 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 --2- 172.21.15.62:0/169726492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f4468105ff0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 -- 172.21.15.62:0/169726492 >> 172.21.15.62:0/169726492 conn(0x7f4468101a00 msgr2=0x7f4468103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 -- 172.21.15.62:0/169726492 shutdown_connections 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.760+0000 7f446f04f640 1 -- 172.21.15.62:0/169726492 wait complete. 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446f04f640 1 Processor -- start 2024-08-16T19:01:14.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446f04f640 1 -- start start 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446f04f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446f04f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4468107630 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446cdeb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446cdeb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36284/0 (socket says 172.21.15.62:36284) 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446cdeb640 1 -- 172.21.15.62:0/871613717 learned_addr learned my addr 172.21.15.62:0/871613717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446cdeb640 1 -- 172.21.15.62:0/871613717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446819da20 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.761+0000 7f446cdeb640 1 --2- 172.21.15.62:0/871613717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4450032ce0 tx=0x7f4450032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4450042070 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44500308c0 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446819dcb0 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4450037040 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44681a09a0 con 0x7f4468105bf0 2024-08-16T19:01:14.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4450045070 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.762+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4450047bd0 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.763+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4468105ff0 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.765+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f445003c830 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.825+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f44681a0b70 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.825+0000 7f4465ffb640 1 -- 172.21.15.62:0/871613717 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f445003ca10 con 0x7f4468105bf0 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 msgr2=0x7f446819d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 --2- 172.21.15.62:0/871613717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4450032ce0 tx=0x7f4450032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 shutdown_connections 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 --2- 172.21.15.62:0/871613717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4468105bf0 0x7f446819d4e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 >> 172.21.15.62:0/871613717 conn(0x7f4468101a00 msgr2=0x7f4468191800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 shutdown_connections 2024-08-16T19:01:14.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:13.826+0000 7f446f04f640 1 -- 172.21.15.62:0/871613717 wait complete. 2024-08-16T19:01:14.341 INFO:teuthology.orchestra.run.smithi062.stdout:Restarting the monitor... 2024-08-16T19:01:14.595 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:14 smithi062 systemd[1]: Stopping Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:01:14.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a[32618]: 2024-08-16T19:01:14.594+0000 7f1dd051b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:01:14.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a[32618]: 2024-08-16T19:01:14.594+0000 7f1dd051b640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-16T19:01:15.169 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:14 smithi062 podman[32865]: 2024-08-16 19:01:14.850673209 +0000 UTC m=+0.351555823 container died 9261a8e35f3032f1e7852aceaf711a8175238c7458d054336b057482b701a111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-16T19:01:15.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 podman[32865]: 2024-08-16 19:01:15.169367279 +0000 UTC m=+0.670249890 container cleanup 9261a8e35f3032f1e7852aceaf711a8175238c7458d054336b057482b701a111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:01:15.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 bash[32865]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a 2024-08-16T19:01:15.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 podman[32878]: 2024-08-16 19:01:15.345430594 +0000 UTC m=+0.494499800 container remove 9261a8e35f3032f1e7852aceaf711a8175238c7458d054336b057482b701a111 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-16T19:01:15.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-16T19:01:15.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 systemd[1]: Stopped Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:01:15.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 systemd[1]: Starting Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:01:16.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 podman[32974]: 2024-08-16 19:01:15.722779885 +0000 UTC m=+0.017783753 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T19:01:16.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:15 smithi062 podman[32974]: 2024-08-16 19:01:15.846387879 +0000 UTC m=+0.141391735 container create aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-16T19:01:16.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 podman[32974]: 2024-08-16 19:01:16.023748165 +0000 UTC m=+0.318752021 container init aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:01:16.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 podman[32974]: 2024-08-16 19:01:16.026986062 +0000 UTC m=+0.321989920 container start aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-16T19:01:16.099 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: set uid:gid to 167:167 (ceph:ceph) 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc), process ceph-mon, pid 2 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: pidfile_write: ignore empty --pid-file 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: load: jerasure load: lrc 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: RocksDB version: 7.9.2 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Git sha 0 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Compile date 2024-08-14 15:56:38 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: DB SUMMARY 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: DB Session ID: TY5JQKU6U3O0OMAE7Y2L 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: CURRENT file: CURRENT 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: IDENTITY file: IDENTITY 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-16T19:01:16.100 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105551 ; 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.error_if_exists: 0 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.create_if_missing: 0 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.paranoid_checks: 1 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.env: 0x558d14ec57a0 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.fs: PosixFileSystem 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.info_log: 0x558d16f59500 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_file_opening_threads: 16 2024-08-16T19:01:16.101 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.statistics: (nil) 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.use_fsync: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_log_file_size: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.keep_log_file_num: 1000 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.recycle_log_file_num: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_fallocate: 1 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_mmap_reads: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_mmap_writes: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.use_direct_reads: 0 2024-08-16T19:01:16.102 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.create_missing_column_families: 0 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.db_log_dir: 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.wal_dir: 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.advise_random_on_open: 1 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.db_write_buffer_size: 0 2024-08-16T19:01:16.103 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.write_buffer_manager: 0x558d16f68dc0 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.rate_limiter: (nil) 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-16T19:01:16.104 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.wal_recovery_mode: 2 2024-08-16T19:01:16.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enable_thread_tracking: 0 2024-08-16T19:01:16.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enable_pipelined_write: 0 2024-08-16T19:01:16.105 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.unordered_write: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.row_cache: None 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.wal_filter: None 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_ingest_behind: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.two_write_queues: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.manual_wal_flush: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.wal_compression: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.atomic_flush: 0 2024-08-16T19:01:16.106 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.log_readahead_size: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.best_efforts_recovery: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.allow_data_in_errors: 0 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.db_host_id: __hostname__ 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_background_jobs: 2 2024-08-16T19:01:16.107 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_background_compactions: -1 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_subcompactions: 1 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_total_wal_size: 0 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-16T19:01:16.108 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_open_files: -1 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bytes_per_sync: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_readahead_size: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_background_flushes: -1 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Compression algorithms supported: 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kZSTD supported: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kXpressCompression supported: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kBZip2Compression supported: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kLZ4Compression supported: 1 2024-08-16T19:01:16.109 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kZlibCompression supported: 1 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kLZ4HCCompression supported: 1 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: kSnappyCompression supported: 1 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.merge_operator: 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_filter: None 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_filter_factory: None 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.sst_partitioner_factory: None 2024-08-16T19:01:16.110 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558d16f59120) 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_type: 0 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_index_type: 0 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_shortening: 1 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: checksum: 4 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: no_block_cache: 0 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache: 0x558d16f4d090 2024-08-16T19:01:16.111 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_options: 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: capacity : 536870912 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_shard_bits : 4 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: strict_capacity_limit : 0 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_cache_compressed: (nil) 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: persistent_cache: (nil) 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size: 4096 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_size_deviation: 10 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_restart_interval: 16 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: index_block_restart_interval: 1 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: metadata_block_size: 4096 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: partition_filters: 0 2024-08-16T19:01:16.112 INFO:journalctl@ceph.mon.a.smithi062.stdout: use_delta_encoding: 1 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: filter_policy: bloomfilter 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: whole_key_filtering: 1 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: verify_compression: 0 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: format_version: 5 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_index_compression: 1 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: block_align: 0 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: max_auto_readahead_size: 262144 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: prepopulate_block_cache: 0 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: initial_auto_readahead_size: 8192 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.write_buffer_size: 33554432 2024-08-16T19:01:16.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_write_buffer_number: 2 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression: NoCompression 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression: Disabled 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.prefix_extractor: nullptr 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.num_levels: 7 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-16T19:01:16.114 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.level: 32767 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.strategy: 0 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.enabled: false 2024-08-16T19:01:16.115 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.target_file_size_base: 67108864 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-16T19:01:16.116 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.arena_block_size: 1048576 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.disable_auto_compactions: 0 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-16T19:01:16.117 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.inplace_update_support: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.bloom_locality: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.max_successive_merges: 0 2024-08-16T19:01:16.118 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.paranoid_file_checks: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.force_consistency_checks: 1 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.report_bg_io_stats: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.ttl: 2592000 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enable_blob_files: false 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.min_blob_size: 0 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_file_size: 268435456 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-16T19:01:16.119 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.blob_file_starting_level: 0 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4f49fc72-5ee4-4d99-a8ea-fb8447b7039e 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1723834876053954, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1723834876055682, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100483, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98368, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14322, "raw_average_key_size": 49, "raw_value_size": 90600, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1723834876, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4f49fc72-5ee4-4d99-a8ea-fb8447b7039e", "db_session_id": "TY5JQKU6U3O0OMAE7Y2L", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: EVENT_LOG_v1 {"time_micros": 1723834876055826, "job": 1, "event": "recovery_finished"} 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-16T19:01:16.120 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558d16f7ae00 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: DB pointer 0x558d17074000 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** DB Stats ** 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-08-16T19:01:16.121 INFO:journalctl@ceph.mon.a.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: L0 2/0 99.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: Sum 2/0 99.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** Compaction Stats [default] ** 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 90.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.122 INFO:journalctl@ceph.mon.a.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Cumulative compaction: 0.00 GB write, 19.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Interval compaction: 0.00 GB write, 19.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache BinnedLRUCache@0x558d16f4d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-16T19:01:16.123 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: starting mon.a rank 0 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???) e1 preinit fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).mds e1 new map 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).mds e1 print_map 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: e1 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: btime 2024-08-16T19:01:02:785450+0000 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: legacy client fscid: -1 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout: No filesystems configured 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-16T19:01:16.124 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-16T19:01:16.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-16T19:01:16.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 bash[32974]: aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 2024-08-16T19:01:16.125 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 systemd[1]: Started Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:01:16.125 INFO:teuthology.orchestra.run.smithi062.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: monmap epoch 1 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: last_changed 2024-08-16T19:00:59.803011+0000 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: min_mon_release 19 (squid) 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: election_strategy: 1 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: fsmap 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: osdmap e1: 0 total, 0 up, 0 in 2024-08-16T19:01:16.355 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:16 smithi062 ceph-mon[32993]: mgrmap e1: no daemons active 2024-08-16T19:01:17.099 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.518+0000 7fb1815f3640 1 Processor -- start 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb1815f3640 1 -- start start 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb1815f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb1815f3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c107dc0 con 0x7fb17c1073f0 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36298/0 (socket says 172.21.15.62:36298) 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.519+0000 7fb17affd640 1 -- 172.21.15.62:0/588317262 learned_addr learned my addr 172.21.15.62:0/588317262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.520+0000 7fb17affd640 1 -- 172.21.15.62:0/588317262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c108550 con 0x7fb17c1073f0 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.521+0000 7fb17affd640 1 --2- 172.21.15.62:0/588317262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009820 tx=0x7fb16c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dd7a0c737bffeb server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.521+0000 7fb17a7fc640 1 -- 172.21.15.62:0/588317262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c037070 con 0x7fb17c1073f0 2024-08-16T19:01:17.104 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb17a7fc640 1 -- 172.21.15.62:0/588317262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb16c032900 con 0x7fb17c1073f0 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb17a7fc640 1 -- 172.21.15.62:0/588317262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c030850 con 0x7fb17c1073f0 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 -- 172.21.15.62:0/588317262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 msgr2=0x7fb17c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 --2- 172.21.15.62:0/588317262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009820 tx=0x7fb16c02a270 comp rx=0 tx=0).stop 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 -- 172.21.15.62:0/588317262 shutdown_connections 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 --2- 172.21.15.62:0/588317262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c1077f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 -- 172.21.15.62:0/588317262 >> 172.21.15.62:0/588317262 conn(0x7fb17c102ba0 msgr2=0x7fb17c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 -- 172.21.15.62:0/588317262 shutdown_connections 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.522+0000 7fb1815f3640 1 -- 172.21.15.62:0/588317262 wait complete. 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb1815f3640 1 Processor -- start 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb1815f3640 1 -- start start 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb1815f3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb1815f3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c10b900 con 0x7fb17c1073f0 2024-08-16T19:01:17.105 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb17affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36300/0 (socket says 172.21.15.62:36300) 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.523+0000 7fb17affd640 1 -- 172.21.15.62:0/36877726 learned_addr learned my addr 172.21.15.62:0/36877726 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb17affd640 1 -- 172.21.15.62:0/36877726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c19b880 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb17affd640 1 --2- 172.21.15.62:0/36877726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009940 tx=0x7fb16c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c042070 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb16c0308c0 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c19bb10 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb16c037040 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c19e800 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb16c045070 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.524+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb16c047bc0 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.525+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c10ba80 con 0x7fb17c1073f0 2024-08-16T19:01:17.106 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.527+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fb16c0327b0 con 0x7fb17c1073f0 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.586+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb17c19edf0 con 0x7fb17c1073f0 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.588+0000 7fb178ff9640 1 -- 172.21.15.62:0/36877726 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb16c03c870 con 0x7fb17c1073f0 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 msgr2=0x7fb17c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 --2- 172.21.15.62:0/36877726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009940 tx=0x7fb16c032ce0 comp rx=0 tx=0).stop 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 shutdown_connections 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 --2- 172.21.15.62:0/36877726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17c1073f0 0x7fb17c19b340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 >> 172.21.15.62:0/36877726 conn(0x7fb17c102ba0 msgr2=0x7fb17c103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 shutdown_connections 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:16.589+0000 7fb1815f3640 1 -- 172.21.15.62:0/36877726 wait complete. 2024-08-16T19:01:17.107 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-16T19:01:17.130 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:01:17.130 INFO:teuthology.orchestra.run.smithi062.stdout:Creating mgr... 2024-08-16T19:01:17.131 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-16T19:01:17.131 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-16T19:01:17.392 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x 2024-08-16T19:01:17.393 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-16T19:01:17.624 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968.target.wants/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@.service. 2024-08-16T19:01:17.867 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:17 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/36877726' entity='client.admin' 2024-08-16T19:01:17.868 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:17 smithi062 systemd[1]: Starting Ceph mgr.x for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:01:18.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:17 smithi062 podman[33203]: 2024-08-16 19:01:17.769894643 +0000 UTC m=+0.017907782 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 2024-08-16T19:01:19.132 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:18 smithi062 podman[33203]: 2024-08-16 19:01:18.859855534 +0000 UTC m=+1.107868667 container create 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-16T19:01:19.132 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 podman[33203]: 2024-08-16 19:01:19.074548105 +0000 UTC m=+1.322561246 container init 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-d863d21, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:01:19.132 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 podman[33203]: 2024-08-16 19:01:19.077890606 +0000 UTC m=+1.325903746 container start 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-16T19:01:19.132 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 bash[33203]: 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 2024-08-16T19:01:19.160 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-16T19:01:19.160 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-08-16T19:01:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-16T19:01:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:01:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-08-16T19:01:19.170 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to enable service . firewalld.service is not available 2024-08-16T19:01:19.180 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-16T19:01:19.180 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr to start... 2024-08-16T19:01:19.189 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr... 2024-08-16T19:01:19.439 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 systemd[1]: Started Ceph mgr.x for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:01:19.439 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:19.216+0000 7f0380059040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:01:19.439 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:19.376+0000 7f0380059040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:19.719 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:19.451+0000 7f0380059040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:01:19.719 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:19.717+0000 7f0380059040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:01:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3893158414' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-16T19:01:19.997 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:19.785+0000 7f0380059040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:01:20.328 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:20.329 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-16T19:01:20.330 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-16T19:01:20.338 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-08-16T19:01:02:785450+0000", 2024-08-16T19:01:20.339 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.340 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-08-16T19:01:02.786113+0000", 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.650+0000 7f50c04e7640 1 Processor -- start 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50c04e7640 1 -- start start 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50c04e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50c04e7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8106bc0 con 0x7f50b81061f0 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50be283640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50be283640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36320/0 (socket says 172.21.15.62:36320) 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.651+0000 7f50be283640 1 -- 172.21.15.62:0/2835611413 learned_addr learned my addr 172.21.15.62:0/2835611413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:20.341 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.652+0000 7f50be283640 1 -- 172.21.15.62:0/2835611413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b81073f0 con 0x7f50b81061f0 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.653+0000 7f50be283640 1 --2- 172.21.15.62:0/2835611413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009820 tx=0x7f50b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8386949062eef414 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.653+0000 7f50bd281640 1 -- 172.21.15.62:0/2835611413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b4037070 con 0x7f50b81061f0 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.653+0000 7f50bd281640 1 -- 172.21.15.62:0/2835611413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50b4032900 con 0x7f50b81061f0 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.653+0000 7f50bd281640 1 -- 172.21.15.62:0/2835611413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b4030850 con 0x7f50b81061f0 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 -- 172.21.15.62:0/2835611413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 msgr2=0x7f50b81065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 --2- 172.21.15.62:0/2835611413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009820 tx=0x7f50b402a270 comp rx=0 tx=0).stop 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 -- 172.21.15.62:0/2835611413 shutdown_connections 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 --2- 172.21.15.62:0/2835611413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b81065f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 -- 172.21.15.62:0/2835611413 >> 172.21.15.62:0/2835611413 conn(0x7f50b8101a00 msgr2=0x7f50b8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 -- 172.21.15.62:0/2835611413 shutdown_connections 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.654+0000 7f50c04e7640 1 -- 172.21.15.62:0/2835611413 wait complete. 2024-08-16T19:01:20.342 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.655+0000 7f50c04e7640 1 Processor -- start 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.655+0000 7f50c04e7640 1 -- start start 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.655+0000 7f50c04e7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.655+0000 7f50c04e7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810a7a0 con 0x7f50b81061f0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.655+0000 7f50be283640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50be283640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36328/0 (socket says 172.21.15.62:36328) 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50be283640 1 -- 172.21.15.62:0/3893158414 learned_addr learned my addr 172.21.15.62:0/3893158414 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50be283640 1 -- 172.21.15.62:0/3893158414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b819e0e0 con 0x7f50b81061f0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50be283640 1 --2- 172.21.15.62:0/3893158414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009940 tx=0x7f50b4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b4042070 con 0x7f50b81061f0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f50b40308c0 con 0x7f50b81061f0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b819e2b0 con 0x7f50b81061f0 2024-08-16T19:01:20.343 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50b4037040 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.656+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f50b4045070 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.657+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b819e650 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.657+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50b40327b0 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.658+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b810a920 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.662+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f50b404d030 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.724+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f50b819e910 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.724+0000 7f50af7fe640 1 -- 172.21.15.62:0/3893158414 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f50b819e910 con 0x7f50b81061f0 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 msgr2=0x7f50b807b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 --2- 172.21.15.62:0/3893158414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009940 tx=0x7f50b4032ce0 comp rx=0 tx=0).stop 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 shutdown_connections 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 --2- 172.21.15.62:0/3893158414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50b81061f0 0x7f50b807b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 >> 172.21.15.62:0/3893158414 conn(0x7f50b8101a00 msgr2=0x7f50b81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:20.344 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 shutdown_connections 2024-08-16T19:01:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:19.725+0000 7f50c04e7640 1 -- 172.21.15.62:0/3893158414 wait complete. 2024-08-16T19:01:20.345 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (1/15)... 2024-08-16T19:01:21.194 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:20.792+0000 7f0380059040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:01:21.194 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:20.912+0000 7f0380059040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:21.194 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:20.971+0000 7f0380059040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:01:21.195 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:21.192+0000 7f0380059040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:01:21.478 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:21.256+0000 7f0380059040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:01:21.478 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:21.325+0000 7f0380059040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:01:21.478 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:21.476+0000 7f0380059040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:22.241 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:21.911+0000 7f0380059040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:01:22.241 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:22.239+0000 7f0380059040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:01:22.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:22.307+0000 7f0380059040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:01:22.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:22.474+0000 7f0380059040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:01:22.887 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:22.813+0000 7f0380059040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:01:22.887 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:22.885+0000 7f0380059040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:01:23.238 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:22 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3708531841' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-16T19:01:23.238 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:23.078+0000 7f0380059040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:01:23.238 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:23.236+0000 7f0380059040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:01:23.399 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:23.401 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:23.401 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:01:23.401 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-08-16T19:01:23.401 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:23.402 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-16T19:01:23.403 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-16T19:01:23.410 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-08-16T19:01:02:785450+0000", 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-16T19:01:23.411 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:23.412 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-08-16T19:01:02.786113+0000", 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.769+0000 7f1a18195640 1 Processor -- start 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.769+0000 7f1a18195640 1 -- start start 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.769+0000 7f1a18195640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.769+0000 7f1a18195640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10106c80 con 0x7f1a101062b0 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.770+0000 7f1a15f31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.770+0000 7f1a15f31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40690/0 (socket says 172.21.15.62:40690) 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.770+0000 7f1a15f31640 1 -- 172.21.15.62:0/3175070064 learned_addr learned my addr 172.21.15.62:0/3175070064 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.770+0000 7f1a15f31640 1 -- 172.21.15.62:0/3175070064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101074b0 con 0x7f1a101062b0 2024-08-16T19:01:23.413 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a15f31640 1 --2- 172.21.15.62:0/3175070064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009820 tx=0x7f1a0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115cdf9d8b468f7d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a14f2f640 1 -- 172.21.15.62:0/3175070064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a0c037070 con 0x7f1a101062b0 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a14f2f640 1 -- 172.21.15.62:0/3175070064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a0c032900 con 0x7f1a101062b0 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a14f2f640 1 -- 172.21.15.62:0/3175070064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a0c030850 con 0x7f1a101062b0 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a18195640 1 -- 172.21.15.62:0/3175070064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 msgr2=0x7f1a101066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.771+0000 7f1a18195640 1 --2- 172.21.15.62:0/3175070064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009820 tx=0x7f1a0c02a270 comp rx=0 tx=0).stop 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 -- 172.21.15.62:0/3175070064 shutdown_connections 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 --2- 172.21.15.62:0/3175070064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a101066b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 -- 172.21.15.62:0/3175070064 >> 172.21.15.62:0/3175070064 conn(0x7f1a10101b00 msgr2=0x7f1a10103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 -- 172.21.15.62:0/3175070064 shutdown_connections 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 -- 172.21.15.62:0/3175070064 wait complete. 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 Processor -- start 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.772+0000 7f1a18195640 1 -- start start 2024-08-16T19:01:23.414 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a18195640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a18195640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1010a860 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a15f31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a15f31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40698/0 (socket says 172.21.15.62:40698) 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a15f31640 1 -- 172.21.15.62:0/3708531841 learned_addr learned my addr 172.21.15.62:0/3708531841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a15f31640 1 -- 172.21.15.62:0/3708531841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1019b8c0 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a15f31640 1 --2- 172.21.15.62:0/3708531841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0097f0 tx=0x7f1a0c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a0c037040 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a0c040070 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1019bb50 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a0c03a390 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.773+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1019e840 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.774+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a0c03a630 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.774+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a0c0459a0 con 0x7f1a101062b0 2024-08-16T19:01:23.415 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.774+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a10106730 con 0x7f1a101062b0 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.776+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f1a0c045460 con 0x7f1a101062b0 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.838+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1a10195310 con 0x7f1a101062b0 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.839+0000 7f1a06ffd640 1 -- 172.21.15.62:0/3708531841 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1a0c03b430 con 0x7f1a101062b0 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 msgr2=0x7f1a1019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 --2- 172.21.15.62:0/3708531841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0097f0 tx=0x7f1a0c030d80 comp rx=0 tx=0).stop 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 shutdown_connections 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 --2- 172.21.15.62:0/3708531841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a101062b0 0x7f1a1019b380 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 >> 172.21.15.62:0/3708531841 conn(0x7f1a10101b00 msgr2=0x7f1a10102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 shutdown_connections 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:22.840+0000 7f1a18195640 1 -- 172.21.15.62:0/3708531841 wait complete. 2024-08-16T19:01:23.416 INFO:teuthology.orchestra.run.smithi062.stdout:mgr not available, waiting (2/15)... 2024-08-16T19:01:23.493 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:23.302+0000 7f0380059040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:01:23.493 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:23.390+0000 7f0380059040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:23.493 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:23.453+0000 7f0380059040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:01:24.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:24.015+0000 7f0380059040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:24.181+0000 7f0380059040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: Activating manager daemon x 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: mgrmap e2: x(active, starting, since 0.00304973s) 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: Manager daemon x is now available 2024-08-16T19:01:24.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:01:24.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' 2024-08-16T19:01:24.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:01:24.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' 2024-08-16T19:01:24.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:24 smithi062 ceph-mon[32993]: from='mgr.14100 172.21.15.62:0/1081248464' entity='mgr.x' 2024-08-16T19:01:26.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:26 smithi062 ceph-mon[32993]: mgrmap e3: x(active, since 1.00729s) 2024-08-16T19:01:26.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:26 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1946310547' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-16T19:01:26.611 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "health": { 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 0 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-16T19:01:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "a" 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-16T19:01:26.613 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-16T19:01:26.614 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "btime": "2024-08-16T19:01:02:785450+0000", 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-16T19:01:26.615 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "restful" 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ], 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "modified": "2024-08-16T19:01:02.786113+0000", 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout }, 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:26.616 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.816+0000 7f4857b94640 1 Processor -- start 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4857b94640 1 -- start start 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4857b94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4857b94640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f485007bd40 con 0x7f4850105c90 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4855930640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4855930640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40766/0 (socket says 172.21.15.62:40766) 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4855930640 1 -- 172.21.15.62:0/1185801401 learned_addr learned my addr 172.21.15.62:0/1185801401 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.817+0000 7f4855930640 1 -- 172.21.15.62:0/1185801401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48501065d0 con 0x7f4850105c90 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.818+0000 7f4855930640 1 --2- 172.21.15.62:0/1185801401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4844009820 tx=0x7f484402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f84ff200b038b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.818+0000 7f485492e640 1 -- 172.21.15.62:0/1185801401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844037070 con 0x7f4850105c90 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.818+0000 7f485492e640 1 -- 172.21.15.62:0/1185801401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4844032900 con 0x7f4850105c90 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.818+0000 7f485492e640 1 -- 172.21.15.62:0/1185801401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844030850 con 0x7f4850105c90 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 -- 172.21.15.62:0/1185801401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 msgr2=0x7f4850106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:26.617 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 --2- 172.21.15.62:0/1185801401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4844009820 tx=0x7f484402a270 comp rx=0 tx=0).stop 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 -- 172.21.15.62:0/1185801401 shutdown_connections 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 --2- 172.21.15.62:0/1185801401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f4850106090 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 -- 172.21.15.62:0/1185801401 >> 172.21.15.62:0/1185801401 conn(0x7f4850101ac0 msgr2=0x7f4850103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 -- 172.21.15.62:0/1185801401 shutdown_connections 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.819+0000 7f4857b94640 1 -- 172.21.15.62:0/1185801401 wait complete. 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4857b94640 1 Processor -- start 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4857b94640 1 -- start start 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4857b94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4857b94640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48501076d0 con 0x7f4850105c90 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4855930640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4855930640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40768/0 (socket says 172.21.15.62:40768) 2024-08-16T19:01:26.618 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4855930640 1 -- 172.21.15.62:0/1946310547 learned_addr learned my addr 172.21.15.62:0/1946310547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4855930640 1 -- 172.21.15.62:0/1946310547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485019da70 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.820+0000 7f4855930640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4844032ce0 tx=0x7f4844032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.821+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844042070 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.821+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f48440308c0 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.821+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485019dd00 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.821+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4844037040 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.821+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a09f0 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f4844045070 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850106090 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f4842ffd640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 0x7f483c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f484406bd70 con 0x7f4850105c90 2024-08-16T19:01:26.619 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f485512f640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 0x7f483c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.822+0000 7f485512f640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 0x7f483c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f484c0098c0 tx=0x7f484c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:25.827+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f484404b170 con 0x7f4850105c90 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.070+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f485002d860 con 0x7f4850105c90 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.070+0000 7f4842ffd640 1 -- 172.21.15.62:0/1946310547 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f484402fd90 con 0x7f4850105c90 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 msgr2=0x7f483c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 0x7f483c03b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f484c0098c0 tx=0x7f484c006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 msgr2=0x7f485019d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4844032ce0 tx=0x7f4844032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 shutdown_connections 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f483c038e60 0x7f483c03b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 --2- 172.21.15.62:0/1946310547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4850105c90 0x7f485019d530 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:26.620 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.072+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 >> 172.21.15.62:0/1946310547 conn(0x7f4850101ac0 msgr2=0x7f48501941f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:26.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.073+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 shutdown_connections 2024-08-16T19:01:26.621 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.073+0000 7f4857b94640 1 -- 172.21.15.62:0/1946310547 wait complete. 2024-08-16T19:01:26.621 INFO:teuthology.orchestra.run.smithi062.stdout:mgr is available 2024-08-16T19:01:27.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:27 smithi062 ceph-mon[32993]: mgrmap e4: x(active, since 2s) 2024-08-16T19:01:27.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:27 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3214769874' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-16T19:01:27.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:27.628 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [global] 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout fsid = bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.62:3300,v1:172.21.15.62:6789] 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout [osd] 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-16T19:01:27.629 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb135e6c640 1 Processor -- start 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb135e6c640 1 -- start start 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb135e6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb135e6c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb13007e0a0 con 0x7fb13007dca0 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40780/0 (socket says 172.21.15.62:40780) 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.942+0000 7fb12f7fe640 1 -- 172.21.15.62:0/3952065745 learned_addr learned my addr 172.21.15.62:0/3952065745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.943+0000 7fb12f7fe640 1 -- 172.21.15.62:0/3952065745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13007c7a0 con 0x7fb13007dca0 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.943+0000 7fb12f7fe640 1 --2- 172.21.15.62:0/3952065745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb120009820 tx=0x7fb12002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5636c200f150c9f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:27.630 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.943+0000 7fb12e7fc640 1 -- 172.21.15.62:0/3952065745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120037070 con 0x7fb13007dca0 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.943+0000 7fb12e7fc640 1 -- 172.21.15.62:0/3952065745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb120032900 con 0x7fb13007dca0 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb12e7fc640 1 -- 172.21.15.62:0/3952065745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120030850 con 0x7fb13007dca0 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 -- 172.21.15.62:0/3952065745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 msgr2=0x7fb13007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3952065745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb120009820 tx=0x7fb12002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 -- 172.21.15.62:0/3952065745 shutdown_connections 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3952065745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13007c260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 -- 172.21.15.62:0/3952065745 >> 172.21.15.62:0/3952065745 conn(0x7fb130102c50 msgr2=0x7fb130105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 -- 172.21.15.62:0/3952065745 shutdown_connections 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.944+0000 7fb135e6c640 1 -- 172.21.15.62:0/3952065745 wait complete. 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb135e6c640 1 Processor -- start 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb135e6c640 1 -- start start 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb135e6c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb135e6c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb13010d8d0 con 0x7fb13007dca0 2024-08-16T19:01:27.631 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb12f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40786/0 (socket says 172.21.15.62:40786) 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.945+0000 7fb12f7fe640 1 -- 172.21.15.62:0/3214769874 learned_addr learned my addr 172.21.15.62:0/3214769874 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb12f7fe640 1 -- 172.21.15.62:0/3214769874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13019da30 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb12f7fe640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097f0 tx=0x7fb120030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120037040 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb120040070 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13019dcc0 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb12003a390 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.946+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a09b0 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb12003a5e0 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb12cff9640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 0x7fb11803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13007c2e0 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb12006c0b0 con 0x7fb13007dca0 2024-08-16T19:01:27.632 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb12effd640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 0x7fb11803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.947+0000 7fb12effd640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 0x7fb11803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb12400ad30 tx=0x7fb1240093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:26.952+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb12006be10 con 0x7fb13007dca0 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.128+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb13019e140 con 0x7fb13007dca0 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.129+0000 7fb12cff9640 1 -- 172.21.15.62:0/3214769874 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fb12003b430 con 0x7fb13007dca0 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.131+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 msgr2=0x7fb11803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.131+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 0x7fb11803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb12400ad30 tx=0x7fb1240093f0 comp rx=0 tx=0).stop 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.131+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 msgr2=0x7fb13019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.131+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097f0 tx=0x7fb120030d80 comp rx=0 tx=0).stop 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 shutdown_connections 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7fb118038f70 0x7fb11803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 --2- 172.21.15.62:0/3214769874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13007dca0 0x7fb13019d4f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 >> 172.21.15.62:0/3214769874 conn(0x7fb130102c50 msgr2=0x7fb1301037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:27.633 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 shutdown_connections 2024-08-16T19:01:27.634 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:27.132+0000 7fb135e6c640 1 -- 172.21.15.62:0/3214769874 wait complete. 2024-08-16T19:01:27.634 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling cephadm module... 2024-08-16T19:01:28.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2775330665' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-16T19:01:28.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setuser ceph since I am not root 2024-08-16T19:01:28.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setgroup ceph since I am not root 2024-08-16T19:01:28.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:28.440+0000 7fd668f90040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:01:28.850 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:28.594+0000 7fd668f90040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:28.850 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:28.665+0000 7fd668f90040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f16101640 1 Processor -- start 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f16101640 1 -- start start 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f16101640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f16101640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10108210 con 0x7f9f10107840 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40802/0 (socket says 172.21.15.62:40802) 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.002+0000 7f9f0f7fe640 1 -- 172.21.15.62:0/1872384221 learned_addr learned my addr 172.21.15.62:0/1872384221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:28.858 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.003+0000 7f9f0f7fe640 1 -- 172.21.15.62:0/1872384221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101064a0 con 0x7f9f10107840 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.003+0000 7f9f0f7fe640 1 --2- 172.21.15.62:0/1872384221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009820 tx=0x7f9f0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145fa472d4cf07b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.003+0000 7f9f0e7fc640 1 -- 172.21.15.62:0/1872384221 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00037070 con 0x7f9f10107840 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.003+0000 7f9f0e7fc640 1 -- 172.21.15.62:0/1872384221 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f00032900 con 0x7f9f10107840 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.003+0000 7f9f0e7fc640 1 -- 172.21.15.62:0/1872384221 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00030850 con 0x7f9f10107840 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 -- 172.21.15.62:0/1872384221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 msgr2=0x7f9f10107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 --2- 172.21.15.62:0/1872384221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009820 tx=0x7f9f0002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 -- 172.21.15.62:0/1872384221 shutdown_connections 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 --2- 172.21.15.62:0/1872384221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10107c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 -- 172.21.15.62:0/1872384221 >> 172.21.15.62:0/1872384221 conn(0x7f9f10101200 msgr2=0x7f9f10103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:28.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 -- 172.21.15.62:0/1872384221 shutdown_connections 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.004+0000 7f9f16101640 1 -- 172.21.15.62:0/1872384221 wait complete. 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f16101640 1 Processor -- start 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f16101640 1 -- start start 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f16101640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f16101640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010a860 con 0x7f9f10107840 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40804/0 (socket says 172.21.15.62:40804) 2024-08-16T19:01:28.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f0f7fe640 1 -- 172.21.15.62:0/2775330665 learned_addr learned my addr 172.21.15.62:0/2775330665 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.005+0000 7f9f0f7fe640 1 -- 172.21.15.62:0/2775330665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f10191a80 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f0f7fe640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f00032ce0 tx=0x7f9f00032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00042070 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f000308c0 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1018fc70 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00037040 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.006+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101901b0 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.007+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9f00045070 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.007+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f10107c40 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.007+0000 7f9f0cff9640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 0x7f9ef803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.007+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f0006c1f0 con 0x7f9f10107840 2024-08-16T19:01:28.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.007+0000 7f9f0effd640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 0x7f9ef803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.008+0000 7f9f0effd640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 0x7f9ef803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f040098c0 tx=0x7f9f04006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.013+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9f0006bc10 con 0x7f9f10107840 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.237+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9f10195330 con 0x7f9f10107840 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.288+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9f00045520 con 0x7f9f10107840 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.290+0000 7f9f0cff9640 1 -- 172.21.15.62:0/2775330665 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9f0003c800 con 0x7f9f10107840 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 msgr2=0x7f9ef803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 0x7f9ef803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f040098c0 tx=0x7f9f04006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:28.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 msgr2=0x7f9f10191540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9f00032ce0 tx=0x7f9f00032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 shutdown_connections 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f9ef8038f20 0x7f9ef803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 --2- 172.21.15.62:0/2775330665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f10107840 0x7f9f10191540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 >> 172.21.15.62:0/2775330665 conn(0x7f9f10101200 msgr2=0x7f9f10102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 shutdown_connections 2024-08-16T19:01:28.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:28.291+0000 7f9f16101640 1 -- 172.21.15.62:0/2775330665 wait complete. 2024-08-16T19:01:29.106 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:28.923+0000 7fd668f90040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:01:29.106 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:28.985+0000 7fd668f90040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:01:29.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:29 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2775330665' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-16T19:01:29.550 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:29 smithi062 ceph-mon[32993]: mgrmap e5: x(active, since 4s) 2024-08-16T19:01:30.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:29 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:29.986+0000 7fd668f90040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:01:30.108 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.107+0000 7fd668f90040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:30.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:30.123 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0396f2d640 1 Processor -- start 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0396f2d640 1 -- start start 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0396f2d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0396f2d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f039007be30 con 0x7f0390107e20 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0394cc9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0394cc9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40850/0 (socket says 172.21.15.62:40850) 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.275+0000 7f0394cc9640 1 -- 172.21.15.62:0/1157823713 learned_addr learned my addr 172.21.15.62:0/1157823713 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:30.124 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.276+0000 7f0394cc9640 1 -- 172.21.15.62:0/1157823713 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039010a750 con 0x7f0390107e20 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.276+0000 7f0394cc9640 1 --2- 172.21.15.62:0/1157823713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0384009820 tx=0x7f038402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63081cbb470dace1 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f038f7fe640 1 -- 172.21.15.62:0/1157823713 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384037070 con 0x7f0390107e20 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f038f7fe640 1 -- 172.21.15.62:0/1157823713 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0384032900 con 0x7f0390107e20 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f038f7fe640 1 -- 172.21.15.62:0/1157823713 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384030850 con 0x7f0390107e20 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 -- 172.21.15.62:0/1157823713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 msgr2=0x7f039010a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 --2- 172.21.15.62:0/1157823713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0384009820 tx=0x7f038402a270 comp rx=0 tx=0).stop 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 -- 172.21.15.62:0/1157823713 shutdown_connections 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 --2- 172.21.15.62:0/1157823713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039010a210 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 -- 172.21.15.62:0/1157823713 >> 172.21.15.62:0/1157823713 conn(0x7f0390101b00 msgr2=0x7f0390103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 -- 172.21.15.62:0/1157823713 shutdown_connections 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.277+0000 7f0396f2d640 1 -- 172.21.15.62:0/1157823713 wait complete. 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0396f2d640 1 Processor -- start 2024-08-16T19:01:30.125 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0396f2d640 1 -- start start 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0396f2d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0396f2d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f039010b780 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0394cc9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0394cc9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40856/0 (socket says 172.21.15.62:40856) 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.278+0000 7f0394cc9640 1 -- 172.21.15.62:0/371048128 learned_addr learned my addr 172.21.15.62:0/371048128 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f0394cc9640 1 -- 172.21.15.62:0/371048128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039019da80 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f0394cc9640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f03840097f0 tx=0x7f0384030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384037040 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0384040070 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039019dd10 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f038403a390 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.279+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a0a00 con 0x7f0390107e20 2024-08-16T19:01:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f038403a5e0 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f038dffb640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 0x7f038003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f038406d1c0 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f038ffff640 1 -- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 msgr2=0x7f038003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0390108280 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.280+0000 7f038ffff640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 0x7f038003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.285+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f038406cf20 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.481+0000 7f038ffff640 1 -- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 msgr2=0x7f038003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.481+0000 7f038ffff640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 0x7f038003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.512+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f03901a0f80 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.512+0000 7f038dffb640 1 -- 172.21.15.62:0/371048128 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f038403a920 con 0x7f0390107e20 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 msgr2=0x7f038003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 0x7f038003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:30.127 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 msgr2=0x7f039019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f03840097f0 tx=0x7f0384030d80 comp rx=0 tx=0).stop 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 shutdown_connections 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0380038fc0 0x7f038003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 --2- 172.21.15.62:0/371048128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0390107e20 0x7f039019d540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 >> 172.21.15.62:0/371048128 conn(0x7f0390101b00 msgr2=0x7f03901025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 shutdown_connections 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:29.515+0000 7f0396f2d640 1 -- 172.21.15.62:0/371048128 wait complete. 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-08-16T19:01:30.128 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 5... 2024-08-16T19:01:30.381 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/371048128' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-16T19:01:30.382 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.181+0000 7fd668f90040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:01:30.707 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.410+0000 7fd668f90040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:01:30.707 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.474+0000 7fd668f90040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:01:30.707 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.549+0000 7fd668f90040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:01:30.707 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:30 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:30.705+0000 7fd668f90040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:31.473 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:31.144+0000 7fd668f90040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:01:31.474 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:31.473+0000 7fd668f90040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:01:31.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:31.542+0000 7fd668f90040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:01:31.747 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:31.697+0000 7fd668f90040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:01:32.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.024+0000 7fd668f90040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:01:32.281 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.088+0000 7fd668f90040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:01:32.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.280+0000 7fd668f90040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:01:32.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.435+0000 7fd668f90040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:01:32.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.502+0000 7fd668f90040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:01:32.589 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.588+0000 7fd668f90040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:32.996 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:32.650+0000 7fd668f90040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:01:33.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:33.212+0000 7fd668f90040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:01:33.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:33.377+0000 7fd668f90040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: Active manager daemon x restarted 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: Activating manager daemon x 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: osdmap e2: 0 total, 0 up, 0 in 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: mgrmap e6: x(active, starting, since 0.00342788s) 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: Manager daemon x is now available 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:33.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:33.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:33.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:01:33.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:33 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:01:34.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:34.939 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.599+0000 7f0574a86640 1 Processor -- start 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.599+0000 7f0574a86640 1 -- start start 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f0574a86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f0574a86640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f057007be30 con 0x7f0570107d90 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f056e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f056e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40862/0 (socket says 172.21.15.62:40862) 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f056e59c640 1 -- 172.21.15.62:0/1286534230 learned_addr learned my addr 172.21.15.62:0/1286534230 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.600+0000 7f056e59c640 1 -- 172.21.15.62:0/1286534230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057010a6c0 con 0x7f0570107d90 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.601+0000 7f056e59c640 1 --2- 172.21.15.62:0/1286534230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0560009820 tx=0x7f056002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77265b6a11df6d1a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:34.940 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f056d59a640 1 -- 172.21.15.62:0/1286534230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0560037070 con 0x7f0570107d90 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f056d59a640 1 -- 172.21.15.62:0/1286534230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0560032900 con 0x7f0570107d90 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f056d59a640 1 -- 172.21.15.62:0/1286534230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0560030850 con 0x7f0570107d90 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f0574a86640 1 -- 172.21.15.62:0/1286534230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 msgr2=0x7f057010a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f0574a86640 1 --2- 172.21.15.62:0/1286534230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0560009820 tx=0x7f056002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f0574a86640 1 -- 172.21.15.62:0/1286534230 shutdown_connections 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f0574a86640 1 --2- 172.21.15.62:0/1286534230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057010a180 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.602+0000 7f0574a86640 1 -- 172.21.15.62:0/1286534230 >> 172.21.15.62:0/1286534230 conn(0x7f0570101ab0 msgr2=0x7f0570103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.603+0000 7f0574a86640 1 -- 172.21.15.62:0/1286534230 shutdown_connections 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.603+0000 7f0574a86640 1 -- 172.21.15.62:0/1286534230 wait complete. 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.603+0000 7f0574a86640 1 Processor -- start 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f0574a86640 1 -- start start 2024-08-16T19:01:34.941 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f0574a86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f0574a86640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f057010b760 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f056e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f056e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40872/0 (socket says 172.21.15.62:40872) 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f056e59c640 1 -- 172.21.15.62:0/3856052731 learned_addr learned my addr 172.21.15.62:0/3856052731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f056e59c640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057019b880 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.604+0000 7f056e59c640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0560031000 tx=0x7f05600326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.605+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0560042070 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.605+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0560032be0 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.605+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057019bb10 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.605+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0560037040 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.605+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057019e800 con 0x7f0570107d90 2024-08-16T19:01:34.942 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0560045070 con 0x7f0570107d90 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f055f7fe640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f055803bb90 con 0x7f0558038fc0 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f056006bf70 con 0x7f0570107d90 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f056dd9b640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 msgr2=0x7f055803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.606+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.807+0000 7f056dd9b640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 msgr2=0x7f055803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:30.807+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:31.208+0000 7f056dd9b640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 msgr2=0x7f055803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:31.208+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:32.009+0000 7f056dd9b640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 msgr2=0x7f055803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/572676055 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:32.009+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:33.380+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f056006aa30 con 0x7f0570107d90 2024-08-16T19:01:34.943 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:33.381+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 msgr2=0x7f055803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:33.381+0000 7f055f7fe640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.382+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f056003c6a0 con 0x7f0570107d90 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.382+0000 7f055f7fe640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 0x7f055803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.382+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0560034ec0 con 0x7f055803c8f0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.382+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 0x7f055803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.383+0000 7f056dd9b640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 0x7f055803ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0564003980 tx=0x7f05640092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.384+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f0560034ec0 con 0x7f055803c8f0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.391+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f05701081f0 con 0x7f055803c8f0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.391+0000 7f055f7fe640 1 -- 172.21.15.62:0/3856052731 <== mgr.14118 v2:172.21.15.62:6800/3198450894 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f05701081f0 con 0x7f055803c8f0 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 msgr2=0x7f055803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 0x7f055803ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0564003980 tx=0x7f05640092b0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 msgr2=0x7f057019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0560031000 tx=0x7f05600326e0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.944 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 shutdown_connections 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f055803c8f0 0x7f055803ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:6800/572676055,v1:172.21.15.62:6801/572676055] conn(0x7f0558038fc0 0x7f055803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 --2- 172.21.15.62:0/3856052731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0570107d90 0x7f057019b340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 >> 172.21.15.62:0/3856052731 conn(0x7f0570101ab0 msgr2=0x7f0570102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 shutdown_connections 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:34.392+0000 7f0574a86640 1 -- 172.21.15.62:0/3856052731 wait complete. 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 5 is available 2024-08-16T19:01:34.945 INFO:teuthology.orchestra.run.smithi062.stdout:Setting orchestrator backend to cephadm... 2024-08-16T19:01:35.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:35 smithi062 ceph-mon[32993]: Found migration_current of "None". Setting to last migration. 2024-08-16T19:01:35.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:35 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:35.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:35 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:35.342 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:35 smithi062 ceph-mon[32993]: mgrmap e7: x(active, since 1.00546s) 2024-08-16T19:01:36.136 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.347+0000 7f268459f640 1 Processor -- start 2024-08-16T19:01:36.136 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.347+0000 7f268459f640 1 -- start start 2024-08-16T19:01:36.136 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268459f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268459f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f267c07bd50 con 0x7f267c107e40 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268233b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268233b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44306/0 (socket says 172.21.15.62:44306) 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268233b640 1 -- 172.21.15.62:0/1087624573 learned_addr learned my addr 172.21.15.62:0/1087624573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.348+0000 7f268233b640 1 -- 172.21.15.62:0/1087624573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c10a770 con 0x7f267c107e40 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.349+0000 7f268233b640 1 --2- 172.21.15.62:0/1087624573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2678009820 tx=0x7f267802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe7aab9e5584ffc server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.349+0000 7f2681339640 1 -- 172.21.15.62:0/1087624573 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678037070 con 0x7f267c107e40 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.349+0000 7f2681339640 1 -- 172.21.15.62:0/1087624573 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2678032900 con 0x7f267c107e40 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.349+0000 7f2681339640 1 -- 172.21.15.62:0/1087624573 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678030850 con 0x7f267c107e40 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 -- 172.21.15.62:0/1087624573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 msgr2=0x7f267c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 --2- 172.21.15.62:0/1087624573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2678009820 tx=0x7f267802a270 comp rx=0 tx=0).stop 2024-08-16T19:01:36.137 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 -- 172.21.15.62:0/1087624573 shutdown_connections 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 --2- 172.21.15.62:0/1087624573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c10a230 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 -- 172.21.15.62:0/1087624573 >> 172.21.15.62:0/1087624573 conn(0x7f267c101a00 msgr2=0x7f267c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 -- 172.21.15.62:0/1087624573 shutdown_connections 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 -- 172.21.15.62:0/1087624573 wait complete. 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.350+0000 7f268459f640 1 Processor -- start 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268459f640 1 -- start start 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268459f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268459f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f267c10b6f0 con 0x7f267c107e40 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268233b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268233b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44314/0 (socket says 172.21.15.62:44314) 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268233b640 1 -- 172.21.15.62:0/1378865901 learned_addr learned my addr 172.21.15.62:0/1378865901 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:36.138 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268233b640 1 -- 172.21.15.62:0/1378865901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c107c00 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268233b640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2678032ce0 tx=0x7f2678032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678042070 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26780308c0 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.351+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c105df0 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.352+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2678037040 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.352+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c106330 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.352+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2678045070 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.353+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c1082a0 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.353+0000 7f266f7fe640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 0x7f266803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.353+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f267806be90 con 0x7f267c107e40 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.353+0000 7f2681b3a640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 0x7f266803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.353+0000 7f2681b3a640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 0x7f266803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26700098c0 tx=0x7f2670006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:36.139 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.358+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f267804b170 con 0x7f267c107e40 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.549+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f267c106be0 con 0x7f2668038e60 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.555+0000 7f266f7fe640 1 -- 172.21.15.62:0/1378865901 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f267c106be0 con 0x7f2668038e60 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 msgr2=0x7f266803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 0x7f266803b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f26700098c0 tx=0x7f2670006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 msgr2=0x7f267c1076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2678032ce0 tx=0x7f2678032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 shutdown_connections 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f2668038e60 0x7f266803b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 --2- 172.21.15.62:0/1378865901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f267c107e40 0x7f267c1076c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 >> 172.21.15.62:0/1378865901 conn(0x7f267c101a00 msgr2=0x7f267c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 shutdown_connections 2024-08-16T19:01:36.140 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:35.557+0000 7f268459f640 1 -- 172.21.15.62:0/1378865901 wait complete. 2024-08-16T19:01:36.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-mon[32993]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-16T19:01:36.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-mon[32993]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-16T19:01:36.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:36.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:36.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:36.142+0000 7fd5d0463640 -1 log_channel(cephadm) log [ERR] : [16/Aug/2024:19:01:36] ENGINE Error in HTTPServer.serve 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Traceback (most recent call last): 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._connections.run(self.expiration_interval) 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._run(expiration_interval) 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:01:36.433 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s = self.context.wrap_socket( 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: return self.sslsocket_class._create( 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.do_handshake() 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._sslobj.do_handshake() 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:01:36.434 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:37.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:37.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:35] ENGINE Bus STARTING 2024-08-16T19:01:37.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:36] ENGINE Serving on http://172.21.15.62:8765 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:36] ENGINE Serving on https://172.21.15.62:7150 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:36] ENGINE Bus STARTED 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:36] ENGINE Error in HTTPServer.serve 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:01:37.247 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:01:37.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:37 smithi062 ceph-mon[32993]: mgrmap e8: x(active, since 2s) 2024-08-16T19:01:37.262 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.511+0000 7f34c2235640 1 Processor -- start 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.511+0000 7f34c2235640 1 -- start start 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.511+0000 7f34c2235640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.511+0000 7f34c2235640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc07bda0 con 0x7f34bc107e30 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.512+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.512+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44324/0 (socket says 172.21.15.62:44324) 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.512+0000 7f34bb7fe640 1 -- 172.21.15.62:0/1302685297 learned_addr learned my addr 172.21.15.62:0/1302685297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.512+0000 7f34bb7fe640 1 -- 172.21.15.62:0/1302685297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc10a760 con 0x7f34bc107e30 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34bb7fe640 1 --2- 172.21.15.62:0/1302685297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009e80 tx=0x7f34ac02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b227ee8ff25430 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:37.263 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34ba7fc640 1 -- 172.21.15.62:0/1302685297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ac036070 con 0x7f34bc107e30 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34ba7fc640 1 -- 172.21.15.62:0/1302685297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f34ac0318c0 con 0x7f34bc107e30 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34ba7fc640 1 -- 172.21.15.62:0/1302685297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ac007ab0 con 0x7f34bc107e30 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34c2235640 1 -- 172.21.15.62:0/1302685297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 msgr2=0x7f34bc10a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34c2235640 1 --2- 172.21.15.62:0/1302685297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009e80 tx=0x7f34ac02a7a0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34c2235640 1 -- 172.21.15.62:0/1302685297 shutdown_connections 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.513+0000 7f34c2235640 1 --2- 172.21.15.62:0/1302685297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc10a220 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.264 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.514+0000 7f34c2235640 1 -- 172.21.15.62:0/1302685297 >> 172.21.15.62:0/1302685297 conn(0x7f34bc101a90 msgr2=0x7f34bc103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.514+0000 7f34c2235640 1 -- 172.21.15.62:0/1302685297 shutdown_connections 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.514+0000 7f34c2235640 1 -- 172.21.15.62:0/1302685297 wait complete. 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.514+0000 7f34c2235640 1 Processor -- start 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.514+0000 7f34c2235640 1 -- start start 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34c2235640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34c2235640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc10b740 con 0x7f34bc107e30 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:37.265 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44340/0 (socket says 172.21.15.62:44340) 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34bb7fe640 1 -- 172.21.15.62:0/101376075 learned_addr learned my addr 172.21.15.62:0/101376075 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34bb7fe640 1 -- 172.21.15.62:0/101376075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc19b860 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34bb7fe640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f34ac02acc0 tx=0x7f34ac031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ac036050 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f34ac037040 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc19baf0 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34ac03d850 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.515+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc19e7e0 con 0x7f34bc107e30 2024-08-16T19:01:37.266 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.516+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f34ac044020 con 0x7f34bc107e30 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.516+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc108290 con 0x7f34bc107e30 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.516+0000 7f34b8ff9640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 0x7f34a803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.517+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f34ac042020 con 0x7f34bc107e30 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.517+0000 7f34baffd640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 0x7f34a803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.517+0000 7f34baffd640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 0x7f34a803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f34a40098c0 tx=0x7f34a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:37.267 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.521+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f34ac04b020 con 0x7f34bc107e30 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.700+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f34bc102d60 con 0x7f34a8038fc0 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.701+0000 7f34b8ff9640 1 -- 172.21.15.62:0/101376075 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f34bc102d60 con 0x7f34a8038fc0 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 msgr2=0x7f34a803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 0x7f34a803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f34a40098c0 tx=0x7f34a4006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 msgr2=0x7f34bc19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:37.268 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f34ac02acc0 tx=0x7f34ac031ca0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 shutdown_connections 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f34a8038fc0 0x7f34a803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 --2- 172.21.15.62:0/101376075 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc107e30 0x7f34bc19b320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 >> 172.21.15.62:0/101376075 conn(0x7f34bc101a90 msgr2=0x7f34bc102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 shutdown_connections 2024-08-16T19:01:37.269 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:36.704+0000 7f34c2235640 1 -- 172.21.15.62:0/101376075 wait complete. 2024-08-16T19:01:37.270 INFO:teuthology.orchestra.run.smithi062.stdout:Generating ssh key... 2024-08-16T19:01:38.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-mon[32993]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:38.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:38.426 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:38.426 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Generating public/private rsa key pair. 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Your identification has been saved in /tmp/tmpejbr_1mc/key 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Your public key has been saved in /tmp/tmpejbr_1mc/key.pub 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: The key fingerprint is: 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: SHA256:12HyvtQN3PoUcvJiTUhdMCLfCcs8L/zc24RkqYp8YOk ceph-bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: The key's randomart image is: 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: +---[RSA 3072]----+ 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | . o o..| 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | = =.o.| 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | . O.o. | 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | *.=.o | 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | S.. =+B+.| 2024-08-16T19:01:38.427 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | +. . OB*.| 2024-08-16T19:01:38.428 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | o . +o*o=| 2024-08-16T19:01:38.428 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | .E..o...+o| 2024-08-16T19:01:38.428 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: | o.. . .o| 2024-08-16T19:01:38.428 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: +----[SHA256]-----+ 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95baefa640 1 Processor -- start 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95baefa640 1 -- start start 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95baefa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95baefa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95b407bd60 con 0x7f95b4105c70 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95b8c96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95b8c96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44346/0 (socket says 172.21.15.62:44346) 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.659+0000 7f95b8c96640 1 -- 172.21.15.62:0/3784445123 learned_addr learned my addr 172.21.15.62:0/3784445123 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.660+0000 7f95b8c96640 1 -- 172.21.15.62:0/3784445123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41065b0 con 0x7f95b4105c70 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.660+0000 7f95b8c96640 1 --2- 172.21.15.62:0/3784445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009820 tx=0x7f95a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0578fdda84a62 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.660+0000 7f95b37fe640 1 -- 172.21.15.62:0/3784445123 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8037070 con 0x7f95b4105c70 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95b37fe640 1 -- 172.21.15.62:0/3784445123 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95a8032900 con 0x7f95b4105c70 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95b37fe640 1 -- 172.21.15.62:0/3784445123 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8030850 con 0x7f95b4105c70 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 -- 172.21.15.62:0/3784445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 msgr2=0x7f95b4106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 --2- 172.21.15.62:0/3784445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009820 tx=0x7f95a802a270 comp rx=0 tx=0).stop 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 -- 172.21.15.62:0/3784445123 shutdown_connections 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 --2- 172.21.15.62:0/3784445123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b4106070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 -- 172.21.15.62:0/3784445123 >> 172.21.15.62:0/3784445123 conn(0x7f95b4101ac0 msgr2=0x7f95b4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 -- 172.21.15.62:0/3784445123 shutdown_connections 2024-08-16T19:01:38.703 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.661+0000 7f95baefa640 1 -- 172.21.15.62:0/3784445123 wait complete. 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95baefa640 1 Processor -- start 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95baefa640 1 -- start start 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95baefa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95baefa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4107530 con 0x7f95b4105c70 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95b8c96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95b8c96640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44362/0 (socket says 172.21.15.62:44362) 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.662+0000 7f95b8c96640 1 -- 172.21.15.62:0/2538709234 learned_addr learned my addr 172.21.15.62:0/2538709234 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95b8c96640 1 -- 172.21.15.62:0/2538709234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b419da50 con 0x7f95b4105c70 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95b8c96640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097f0 tx=0x7f95a8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a8037040 con 0x7f95b4105c70 2024-08-16T19:01:38.704 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95a8040070 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a803a390 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b419dce0 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.663+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a09d0 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.664+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f95a803b430 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.664+0000 7f95b1ffb640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 0x7f95a403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.664+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b4107c00 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.664+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95a806c490 con 0x7f95b4105c70 2024-08-16T19:01:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.664+0000 7f95b3fff640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 0x7f95a403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.665+0000 7f95b3fff640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 0x7f95a403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95a00098c0 tx=0x7f95a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.669+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f95a803a5e0 con 0x7f95b4105c70 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:37.849+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f95b402d950 con 0x7f95a4038f70 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.132+0000 7f95b1ffb640 1 -- 172.21.15.62:0/2538709234 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95b402d950 con 0x7f95a4038f70 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 msgr2=0x7f95a403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 0x7f95a403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95a00098c0 tx=0x7f95a0006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 msgr2=0x7f95b419d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:38.706 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95a80097f0 tx=0x7f95a8030d80 comp rx=0 tx=0).stop 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 shutdown_connections 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f95a4038f70 0x7f95a403b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 --2- 172.21.15.62:0/2538709234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b4105c70 0x7f95b419d510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 >> 172.21.15.62:0/2538709234 conn(0x7f95b4101ac0 msgr2=0x7f95b41941d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 shutdown_connections 2024-08-16T19:01:38.707 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:38.134+0000 7f95baefa640 1 -- 172.21.15.62:0/2538709234 wait complete. 2024-08-16T19:01:39.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:39 smithi062 ceph-mon[32993]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:39.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:39 smithi062 ceph-mon[32993]: Generating ssh key... 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.051+0000 7f81b3f9c640 1 Processor -- start 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b3f9c640 1 -- start start 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b3f9c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b3f9c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac07be30 con 0x7f81ac107d60 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b1d38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b1d38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44368/0 (socket says 172.21.15.62:44368) 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b1d38640 1 -- 172.21.15.62:0/222092195 learned_addr learned my addr 172.21.15.62:0/222092195 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:39.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.052+0000 7f81b1d38640 1 -- 172.21.15.62:0/222092195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac10a690 con 0x7f81ac107d60 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.053+0000 7f81b1d38640 1 --2- 172.21.15.62:0/222092195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009820 tx=0x7f81a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8c89e9f5360f62 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.053+0000 7f81b0d36640 1 -- 172.21.15.62:0/222092195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81a8037070 con 0x7f81ac107d60 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.053+0000 7f81b0d36640 1 -- 172.21.15.62:0/222092195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81a8032900 con 0x7f81ac107d60 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.053+0000 7f81b0d36640 1 -- 172.21.15.62:0/222092195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81a8030850 con 0x7f81ac107d60 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- 172.21.15.62:0/222092195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 msgr2=0x7f81ac10a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/222092195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009820 tx=0x7f81a802a270 comp rx=0 tx=0).stop 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- 172.21.15.62:0/222092195 shutdown_connections 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/222092195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac10a150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- 172.21.15.62:0/222092195 >> 172.21.15.62:0/222092195 conn(0x7f81ac101a00 msgr2=0x7f81ac103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- 172.21.15.62:0/222092195 shutdown_connections 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- 172.21.15.62:0/222092195 wait complete. 2024-08-16T19:01:39.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 Processor -- start 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.054+0000 7f81b3f9c640 1 -- start start 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b3f9c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b3f9c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10b730 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b1d38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b1d38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44378/0 (socket says 172.21.15.62:44378) 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b1d38640 1 -- 172.21.15.62:0/104725269 learned_addr learned my addr 172.21.15.62:0/104725269 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b1d38640 1 -- 172.21.15.62:0/104725269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac19b830 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b1d38640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81a8032ce0 tx=0x7f81a8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81a8042070 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81a80308c0 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac19bac0 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81a8037040 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.055+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac19e7b0 con 0x7f81ac107d60 2024-08-16T19:01:39.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.056+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f81a8045070 con 0x7f81ac107d60 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.056+0000 7f819effd640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 0x7f819803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.057+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81a806d0b0 con 0x7f81ac107d60 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.057+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac1081c0 con 0x7f81ac107d60 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.057+0000 7f81b1537640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 0x7f819803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.057+0000 7f81b1537640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 0x7f819803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098c0 tx=0x7f81a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.061+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f81a8045520 con 0x7f81ac107d60 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.238+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f81ac102cc0 con 0x7f8198038f70 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.239+0000 7f819effd640 1 -- 172.21.15.62:0/104725269 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f81ac102cc0 con 0x7f8198038f70 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 msgr2=0x7f819803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 0x7f819803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81a00098c0 tx=0x7f81a0006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 msgr2=0x7f81ac19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81a8032ce0 tx=0x7f81a8032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:39.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 shutdown_connections 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8198038f70 0x7f819803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 --2- 172.21.15.62:0/104725269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81ac107d60 0x7f81ac19b2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 >> 172.21.15.62:0/104725269 conn(0x7f81ac101a00 msgr2=0x7f81ac1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 shutdown_connections 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:39.242+0000 7f81b3f9c640 1 -- 172.21.15.62:0/104725269 wait complete. 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:Adding key to root@localhost authorized_keys... 2024-08-16T19:01:39.805 INFO:teuthology.orchestra.run.smithi062.stdout:Adding host smithi062... 2024-08-16T19:01:40.340 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:40 smithi062 ceph-mon[32993]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:41.367 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:41 smithi062 ceph-mon[32993]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:42.588 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:42 smithi062 ceph-mon[32993]: Deploying cephadm binary to smithi062 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Added host 'smithi062' with addr '172.21.15.62' 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b6de0e640 1 Processor -- start 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b6de0e640 1 -- start start 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b6de0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b6de0e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6807e0c0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44388/0 (socket says 172.21.15.62:44388) 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.149+0000 7f1b677fe640 1 -- 172.21.15.62:0/167935758 learned_addr learned my addr 172.21.15.62:0/167935758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.150+0000 7f1b677fe640 1 -- 172.21.15.62:0/167935758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6807c7c0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.801 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.150+0000 7f1b677fe640 1 --2- 172.21.15.62:0/167935758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009e80 tx=0x7f1b5802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd2d292f4b57d28 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b667fc640 1 -- 172.21.15.62:0/167935758 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b58036070 con 0x7f1b6807dcc0 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b667fc640 1 -- 172.21.15.62:0/167935758 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b580318c0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b667fc640 1 -- 172.21.15.62:0/167935758 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b58007ab0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 -- 172.21.15.62:0/167935758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 msgr2=0x7f1b6807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/167935758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009e80 tx=0x7f1b5802a7a0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 -- 172.21.15.62:0/167935758 shutdown_connections 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/167935758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6807c280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.802 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 -- 172.21.15.62:0/167935758 >> 172.21.15.62:0/167935758 conn(0x7f1b68102c50 msgr2=0x7f1b68105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 -- 172.21.15.62:0/167935758 shutdown_connections 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.151+0000 7f1b6de0e640 1 -- 172.21.15.62:0/167935758 wait complete. 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.152+0000 7f1b6de0e640 1 Processor -- start 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.152+0000 7f1b6de0e640 1 -- start start 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.152+0000 7f1b6de0e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.152+0000 7f1b6de0e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6810d900 con 0x7f1b6807dcc0 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44400/0 (socket says 172.21.15.62:44400) 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b677fe640 1 -- 172.21.15.62:0/3717743524 learned_addr learned my addr 172.21.15.62:0/3717743524 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b677fe640 1 -- 172.21.15.62:0/3717743524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6819da20 con 0x7f1b6807dcc0 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b677fe640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b5802acc0 tx=0x7f1b58031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b58036050 con 0x7f1b6807dcc0 2024-08-16T19:01:44.803 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b58037040 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6819dcb0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5803d850 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.153+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b681a09a0 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.154+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1b58044020 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.154+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6807c300 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.155+0000 7f1b64ff9640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 0x7f1b5003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.155+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b58042020 con 0x7f1b6807dcc0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.155+0000 7f1b66ffd640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 0x7f1b5003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.155+0000 7f1b66ffd640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 0x7f1b5003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0098c0 tx=0x7f1b5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.159+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1b5804b020 con 0x7f1b6807dcc0 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:40.340+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "addr": "172.21.15.62", "target": ["mon-mgr", ""]}) -- 0x7f1b68103eb0 con 0x7f1b50038fc0 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.241+0000 7f1b64ff9640 1 -- 172.21.15.62:0/3717743524 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f1b68103eb0 con 0x7f1b50038fc0 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 msgr2=0x7f1b5003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 0x7f1b5003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0098c0 tx=0x7f1b5c006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 msgr2=0x7f1b6819d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1b5802acc0 tx=0x7f1b58031ca0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 shutdown_connections 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1b50038fc0 0x7f1b5003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 --2- 172.21.15.62:0/3717743524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b6807dcc0 0x7f1b6819d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 >> 172.21.15.62:0/3717743524 conn(0x7f1b68102c50 msgr2=0x7f1b681037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 shutdown_connections 2024-08-16T19:01:44.805 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:44.244+0000 7f1b6de0e640 1 -- 172.21.15.62:0/3717743524 wait complete. 2024-08-16T19:01:44.806 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mon service... 2024-08-16T19:01:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:45 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:45 smithi062 ceph-mon[32993]: Added host smithi062 2024-08-16T19:01:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:45 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:46.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-16T19:01:46.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.194+0000 7f899f722640 1 Processor -- start 2024-08-16T19:01:46.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.194+0000 7f899f722640 1 -- start start 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.194+0000 7f899f722640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.194+0000 7f899f722640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8998107d20 con 0x7f89981073e0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.195+0000 7f899d4be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.195+0000 7f899d4be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43238/0 (socket says 172.21.15.62:43238) 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.195+0000 7f899d4be640 1 -- 172.21.15.62:0/3975215954 learned_addr learned my addr 172.21.15.62:0/3975215954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.195+0000 7f899d4be640 1 -- 172.21.15.62:0/3975215954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8998107ea0 con 0x7f89981073e0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f899d4be640 1 --2- 172.21.15.62:0/3975215954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8994009820 tx=0x7f899402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7652d1d298e7b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f898ffff640 1 -- 172.21.15.62:0/3975215954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8994037070 con 0x7f89981073e0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f898ffff640 1 -- 172.21.15.62:0/3975215954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8994032900 con 0x7f89981073e0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f898ffff640 1 -- 172.21.15.62:0/3975215954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8994030850 con 0x7f89981073e0 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f899f722640 1 -- 172.21.15.62:0/3975215954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 msgr2=0x7f89981077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.196+0000 7f899f722640 1 --2- 172.21.15.62:0/3975215954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8994009820 tx=0x7f899402a270 comp rx=0 tx=0).stop 2024-08-16T19:01:46.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 -- 172.21.15.62:0/3975215954 shutdown_connections 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 --2- 172.21.15.62:0/3975215954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f89981077e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 -- 172.21.15.62:0/3975215954 >> 172.21.15.62:0/3975215954 conn(0x7f8998102c50 msgr2=0x7f8998105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 -- 172.21.15.62:0/3975215954 shutdown_connections 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 -- 172.21.15.62:0/3975215954 wait complete. 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 Processor -- start 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.197+0000 7f899f722640 1 -- start start 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899f722640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899f722640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f899810b800 con 0x7f89981073e0 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899d4be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899d4be640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43252/0 (socket says 172.21.15.62:43252) 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899d4be640 1 -- 172.21.15.62:0/2969812161 learned_addr learned my addr 172.21.15.62:0/2969812161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:46.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899d4be640 1 -- 172.21.15.62:0/2969812161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819da60 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899d4be640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8994032ce0 tx=0x7f8994032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8994042070 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89940308c0 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8994037040 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819dcf0 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.198+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a09e0 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.199+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8994045070 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.199+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8998107860 con 0x7f89981073e0 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.199+0000 7f898e7fc640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 0x7f898403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:46.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.200+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f899406bfd0 con 0x7f89981073e0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.200+0000 7f899ccbd640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 0x7f898403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.200+0000 7f899ccbd640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 0x7f898403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89880098c0 tx=0x7f8988006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.204+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f899403c760 con 0x7f89981073e0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.397+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8998103de0 con 0x7f8984038fc0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.402+0000 7f898e7fc640 1 -- 172.21.15.62:0/2969812161 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8998103de0 con 0x7f8984038fc0 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 msgr2=0x7f898403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 0x7f898403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89880098c0 tx=0x7f8988006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 msgr2=0x7f899819d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8994032ce0 tx=0x7f8994032d10 comp rx=0 tx=0).stop 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 shutdown_connections 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f8984038fc0 0x7f898403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 --2- 172.21.15.62:0/2969812161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89981073e0 0x7f899819d520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 >> 172.21.15.62:0/2969812161 conn(0x7f8998102c50 msgr2=0x7f89981036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:46.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 shutdown_connections 2024-08-16T19:01:46.172 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:45.404+0000 7f899f722640 1 -- 172.21.15.62:0/2969812161 wait complete. 2024-08-16T19:01:46.172 INFO:teuthology.orchestra.run.smithi062.stdout:Deploying unmanaged mgr service... 2024-08-16T19:01:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:46 smithi062 ceph-mon[32993]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:46 smithi062 ceph-mon[32993]: Saving service mon spec with placement count:5 2024-08-16T19:01:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:46 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:48.039 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-16T19:01:48.039 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9563c6d640 1 Processor -- start 2024-08-16T19:01:48.039 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9563c6d640 1 -- start start 2024-08-16T19:01:48.039 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9563c6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9563c6d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f955c07be30 con 0x7f955c107d60 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9561a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9561a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43256/0 (socket says 172.21.15.62:43256) 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.807+0000 7f9561a09640 1 -- 172.21.15.62:0/1767395580 learned_addr learned my addr 172.21.15.62:0/1767395580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.808+0000 7f9561a09640 1 -- 172.21.15.62:0/1767395580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c10a690 con 0x7f955c107d60 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9561a09640 1 --2- 172.21.15.62:0/1767395580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9558009820 tx=0x7f955802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa69d36489eb426 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9560a07640 1 -- 172.21.15.62:0/1767395580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9558037070 con 0x7f955c107d60 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9560a07640 1 -- 172.21.15.62:0/1767395580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9558032900 con 0x7f955c107d60 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9560a07640 1 -- 172.21.15.62:0/1767395580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9558030850 con 0x7f955c107d60 2024-08-16T19:01:48.040 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9563c6d640 1 -- 172.21.15.62:0/1767395580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 msgr2=0x7f955c10a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.809+0000 7f9563c6d640 1 --2- 172.21.15.62:0/1767395580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9558009820 tx=0x7f955802a270 comp rx=0 tx=0).stop 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- 172.21.15.62:0/1767395580 shutdown_connections 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 --2- 172.21.15.62:0/1767395580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c10a150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- 172.21.15.62:0/1767395580 >> 172.21.15.62:0/1767395580 conn(0x7f955c101a00 msgr2=0x7f955c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- 172.21.15.62:0/1767395580 shutdown_connections 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- 172.21.15.62:0/1767395580 wait complete. 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 Processor -- start 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- start start 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.810+0000 7f9563c6d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f955c10b730 con 0x7f955c107d60 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9561a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9561a09640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43258/0 (socket says 172.21.15.62:43258) 2024-08-16T19:01:48.041 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9561a09640 1 -- 172.21.15.62:0/725629989 learned_addr learned my addr 172.21.15.62:0/725629989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9561a09640 1 -- 172.21.15.62:0/725629989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c107b20 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9561a09640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95580097f0 tx=0x7f9558030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9558037040 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9558040070 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f955803a390 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.811+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c105d10 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.812+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1061f0 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.812+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f955803a5e0 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.813+0000 7f9552ffd640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 0x7f954803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.813+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f955806c5c0 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.813+0000 7f9561208640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 0x7f954803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.813+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c1081c0 con 0x7f955c107d60 2024-08-16T19:01:48.042 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.813+0000 7f9561208640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 0x7f954803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098c0 tx=0x7f954c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:46.818+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f955803a920 con 0x7f955c107d60 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.013+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f955c106960 con 0x7f9548038fc0 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.017+0000 7f9552ffd640 1 -- 172.21.15.62:0/725629989 <== mgr.14118 v2:172.21.15.62:6800/3198450894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f955c106960 con 0x7f9548038fc0 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 msgr2=0x7f954803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 0x7f954803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098c0 tx=0x7f954c006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 msgr2=0x7f955c1075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95580097f0 tx=0x7f9558030d80 comp rx=0 tx=0).stop 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 shutdown_connections 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f9548038fc0 0x7f954803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 --2- 172.21.15.62:0/725629989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f955c107d60 0x7f955c1075e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.019+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 >> 172.21.15.62:0/725629989 conn(0x7f955c101a00 msgr2=0x7f955c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:48.043 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.020+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 shutdown_connections 2024-08-16T19:01:48.044 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:47.020+0000 7f9563c6d640 1 -- 172.21.15.62:0/725629989 wait complete. 2024-08-16T19:01:48.158 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:48 smithi062 ceph-mon[32993]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:01:48.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:48 smithi062 ceph-mon[32993]: Saving service mgr spec with placement count:2 2024-08-16T19:01:48.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:48 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:48.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:48 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:48.159 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:48 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:49.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:49 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/120962733' entity='client.admin' 2024-08-16T19:01:49.851 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:49 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.442+0000 7f30dd129640 1 Processor -- start 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.442+0000 7f30dd129640 1 -- start start 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.442+0000 7f30dd129640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.442+0000 7f30dd129640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30d807e0c0 con 0x7f30d807dcc0 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.442+0000 7f30d6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:49.859 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.443+0000 7f30d6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43264/0 (socket says 172.21.15.62:43264) 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.443+0000 7f30d6d9d640 1 -- 172.21.15.62:0/1603051358 learned_addr learned my addr 172.21.15.62:0/1603051358 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.443+0000 7f30d6d9d640 1 -- 172.21.15.62:0/1603051358 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d807c7c0 con 0x7f30d807dcc0 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.443+0000 7f30d6d9d640 1 --2- 172.21.15.62:0/1603051358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009820 tx=0x7f30c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7db2c76ed1f1ce4 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30d659c640 1 -- 172.21.15.62:0/1603051358 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c8037070 con 0x7f30d807dcc0 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30d659c640 1 -- 172.21.15.62:0/1603051358 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8032920 con 0x7f30d807dcc0 2024-08-16T19:01:49.860 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30d659c640 1 -- 172.21.15.62:0/1603051358 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c8030810 con 0x7f30d807dcc0 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 -- 172.21.15.62:0/1603051358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 msgr2=0x7f30d807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 --2- 172.21.15.62:0/1603051358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009820 tx=0x7f30c802a270 comp rx=0 tx=0).stop 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 -- 172.21.15.62:0/1603051358 shutdown_connections 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 --2- 172.21.15.62:0/1603051358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d807c280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 -- 172.21.15.62:0/1603051358 >> 172.21.15.62:0/1603051358 conn(0x7f30d8102c50 msgr2=0x7f30d8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:49.861 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 -- 172.21.15.62:0/1603051358 shutdown_connections 2024-08-16T19:01:49.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.444+0000 7f30dd129640 1 -- 172.21.15.62:0/1603051358 wait complete. 2024-08-16T19:01:49.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.445+0000 7f30dd129640 1 Processor -- start 2024-08-16T19:01:49.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.445+0000 7f30dd129640 1 -- start start 2024-08-16T19:01:49.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.445+0000 7f30dd129640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:49.862 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.445+0000 7f30dd129640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30d810d900 con 0x7f30d807dcc0 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.445+0000 7f30d6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43268/0 (socket says 172.21.15.62:43268) 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d6d9d640 1 -- 172.21.15.62:0/120962733 learned_addr learned my addr 172.21.15.62:0/120962733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d6d9d640 1 -- 172.21.15.62:0/120962733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d819da30 con 0x7f30d807dcc0 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d6d9d640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30c80097f0 tx=0x7f30c80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:49.863 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c8037040 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8030cc0 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d819dcc0 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c803a6b0 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.446+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81a09b0 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.447+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f30c803b3f0 con 0x7f30d807dcc0 2024-08-16T19:01:49.864 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.447+0000 7f30d4d99640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 0x7f30c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:49.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.447+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d807c300 con 0x7f30d807dcc0 2024-08-16T19:01:49.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.447+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f30c806c510 con 0x7f30d807dcc0 2024-08-16T19:01:49.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.447+0000 7f30cffff640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 0x7f30c403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:49.865 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.448+0000 7f30cffff640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 0x7f30c403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f30c00098c0 tx=0x7f30c0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:49.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.453+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f30c80371e0 con 0x7f30d807dcc0 2024-08-16T19:01:49.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.635+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f30d81a0f90 con 0x7f30d807dcc0 2024-08-16T19:01:49.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.638+0000 7f30d4d99640 1 -- 172.21.15.62:0/120962733 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f30c806bea0 con 0x7f30d807dcc0 2024-08-16T19:01:49.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 msgr2=0x7f30c403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:49.866 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 0x7f30c403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f30c00098c0 tx=0x7f30c0006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 msgr2=0x7f30d819d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30c80097f0 tx=0x7f30c80327b0 comp rx=0 tx=0).stop 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 shutdown_connections 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f30c4038fc0 0x7f30c403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 --2- 172.21.15.62:0/120962733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30d807dcc0 0x7f30d819d4f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:49.867 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 >> 172.21.15.62:0/120962733 conn(0x7f30d8102c50 msgr2=0x7f30d81037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:49.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 shutdown_connections 2024-08-16T19:01:49.868 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:48.640+0000 7f30dd129640 1 -- 172.21.15.62:0/120962733 wait complete. 2024-08-16T19:01:51.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a2ec0640 1 Processor -- start 2024-08-16T19:01:51.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a2ec0640 1 -- start start 2024-08-16T19:01:51.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a2ec0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:51.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a2ec0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f699c106bc0 con 0x7f699c1061f0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a0c5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a0c5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43274/0 (socket says 172.21.15.62:43274) 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a0c5c640 1 -- 172.21.15.62:0/3813548598 learned_addr learned my addr 172.21.15.62:0/3813548598 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.262+0000 7f69a0c5c640 1 -- 172.21.15.62:0/3813548598 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1073f0 con 0x7f699c1061f0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.263+0000 7f69a0c5c640 1 --2- 172.21.15.62:0/3813548598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6990009820 tx=0x7f699002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a6edd2b7f78722 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.263+0000 7f699b7fe640 1 -- 172.21.15.62:0/3813548598 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990037070 con 0x7f699c1061f0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.263+0000 7f699b7fe640 1 -- 172.21.15.62:0/3813548598 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6990032920 con 0x7f699c1061f0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.263+0000 7f699b7fe640 1 -- 172.21.15.62:0/3813548598 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990030810 con 0x7f699c1061f0 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3813548598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 msgr2=0x7f699c1065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3813548598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6990009820 tx=0x7f699002a270 comp rx=0 tx=0).stop 2024-08-16T19:01:51.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3813548598 shutdown_connections 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3813548598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c1065f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3813548598 >> 172.21.15.62:0/3813548598 conn(0x7f699c101a00 msgr2=0x7f699c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3813548598 shutdown_connections 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.264+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3813548598 wait complete. 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a2ec0640 1 Processor -- start 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a2ec0640 1 -- start start 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a2ec0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a2ec0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f699c10a7a0 con 0x7f699c1061f0 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a0c5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a0c5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43288/0 (socket says 172.21.15.62:43288) 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a0c5c640 1 -- 172.21.15.62:0/3358965979 learned_addr learned my addr 172.21.15.62:0/3358965979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a0c5c640 1 -- 172.21.15.62:0/3358965979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c19b8c0 con 0x7f699c1061f0 2024-08-16T19:01:51.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.265+0000 7f69a0c5c640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6990030560 tx=0x7f6990032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.266+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6990037040 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.266+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c19bb50 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.266+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6990040070 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.266+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f699003a740 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.266+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c19e840 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.267+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f699003a9b0 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.267+0000 7f6999ffb640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 0x7f698c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.267+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f699006c5e0 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.267+0000 7f699bfff640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 0x7f698c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.267+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c10a920 con 0x7f699c1061f0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.268+0000 7f699bfff640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 0x7f698c03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69880098c0 tx=0x7f6988006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:51.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.272+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6990041270 con 0x7f699c1061f0 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.465+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f699c02d6c0 con 0x7f699c1061f0 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.468+0000 7f6999ffb640 1 -- 172.21.15.62:0/3358965979 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f69900328f0 con 0x7f699c1061f0 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 msgr2=0x7f698c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 0x7f698c03b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f69880098c0 tx=0x7f6988006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 msgr2=0x7f699c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6990030560 tx=0x7f6990032670 comp rx=0 tx=0).stop 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 shutdown_connections 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f698c038f70 0x7f698c03b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 --2- 172.21.15.62:0/3358965979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1061f0 0x7f699c19b380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.470+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 >> 172.21.15.62:0/3358965979 conn(0x7f699c101a00 msgr2=0x7f699c102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.471+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 shutdown_connections 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:50.471+0000 7f69a2ec0640 1 -- 172.21.15.62:0/3358965979 wait complete. 2024-08-16T19:01:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:Enabling the dashboard module... 2024-08-16T19:01:51.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:51 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3358965979' entity='client.admin' 2024-08-16T19:01:51.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:51 smithi062 ceph-mon[32993]: from='mgr.14118 172.21.15.62:0/138251514' entity='mgr.x' 2024-08-16T19:01:52.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/126894104' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-16T19:01:52.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setuser ceph since I am not root 2024-08-16T19:01:52.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setgroup ceph since I am not root 2024-08-16T19:01:52.747 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:52.638+0000 7fae922ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:01:53.119 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:52.791+0000 7fae922ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:53.119 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:52.865+0000 7fae922ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:01:53.119 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:53.118+0000 7fae922ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.460+0000 7fa6ff615640 1 Processor -- start 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6ff615640 1 -- start start 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6ff615640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6ff615640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8107d20 con 0x7fa6f81073e0 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6fd3b1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6fd3b1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43662/0 (socket says 172.21.15.62:43662) 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.461+0000 7fa6fd3b1640 1 -- 172.21.15.62:0/1082112085 learned_addr learned my addr 172.21.15.62:0/1082112085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.462+0000 7fa6fd3b1640 1 -- 172.21.15.62:0/1082112085 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8107ea0 con 0x7fa6f81073e0 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.462+0000 7fa6fd3b1640 1 --2- 172.21.15.62:0/1082112085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009e80 tx=0x7fa6e802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ad744644a6cc91 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.462+0000 7fa6f7fff640 1 -- 172.21.15.62:0/1082112085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e8036070 con 0x7fa6f81073e0 2024-08-16T19:01:53.167 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.462+0000 7fa6f7fff640 1 -- 172.21.15.62:0/1082112085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e80318e0 con 0x7fa6f81073e0 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.462+0000 7fa6f7fff640 1 -- 172.21.15.62:0/1082112085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e8007a00 con 0x7fa6f81073e0 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 -- 172.21.15.62:0/1082112085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 msgr2=0x7fa6f81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 --2- 172.21.15.62:0/1082112085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009e80 tx=0x7fa6e802a7a0 comp rx=0 tx=0).stop 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 -- 172.21.15.62:0/1082112085 shutdown_connections 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 --2- 172.21.15.62:0/1082112085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f81077e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 -- 172.21.15.62:0/1082112085 >> 172.21.15.62:0/1082112085 conn(0x7fa6f8102c50 msgr2=0x7fa6f8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 -- 172.21.15.62:0/1082112085 shutdown_connections 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.463+0000 7fa6ff615640 1 -- 172.21.15.62:0/1082112085 wait complete. 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6ff615640 1 Processor -- start 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6ff615640 1 -- start start 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6ff615640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6ff615640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f810b800 con 0x7fa6f81073e0 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6fd3b1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:53.168 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6fd3b1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43674/0 (socket says 172.21.15.62:43674) 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6fd3b1640 1 -- 172.21.15.62:0/126894104 learned_addr learned my addr 172.21.15.62:0/126894104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6fd3b1640 1 -- 172.21.15.62:0/126894104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f819b840 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.464+0000 7fa6fd3b1640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8031cf0 tx=0x7fa6e8031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.465+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e8036050 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.465+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e8042070 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.465+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f819bad0 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.465+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e803d9b0 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.465+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f819e7c0 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa6e803db50 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6f67fc640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 0x7fa6e003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6e806d340 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f8107860 con 0x7fa6f81073e0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6fcbb0640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 0x7fa6e003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:53.169 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.466+0000 7fa6fcbb0640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 0x7fa6e003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098c0 tx=0x7fa6ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.471+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa6e8044310 con 0x7fa6f81073e0 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:51.708+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa6f819ec80 con 0x7fa6f81073e0 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.475+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa6e8045520 con 0x7fa6f81073e0 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.476+0000 7fa6f67fc640 1 -- 172.21.15.62:0/126894104 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa6e8039300 con 0x7fa6f81073e0 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 msgr2=0x7fa6e003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 0x7fa6e003b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0098c0 tx=0x7fa6ec006db0 comp rx=0 tx=0).stop 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 msgr2=0x7fa6f819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8031cf0 tx=0x7fa6e8031d20 comp rx=0 tx=0).stop 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 shutdown_connections 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7fa6e0038fc0 0x7fa6e003b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 --2- 172.21.15.62:0/126894104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f81073e0 0x7fa6f819b300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 >> 172.21.15.62:0/126894104 conn(0x7fa6f8102c50 msgr2=0x7fa6f8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:53.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 shutdown_connections 2024-08-16T19:01:53.171 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:52.478+0000 7fa6ff615640 1 -- 172.21.15.62:0/126894104 wait complete. 2024-08-16T19:01:53.400 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:53.184+0000 7fae922ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:01:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:53 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/126894104' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-16T19:01:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:53 smithi062 ceph-mon[32993]: mgrmap e9: x(active, since 19s) 2024-08-16T19:01:54.369 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.187+0000 7fae922ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:01:54.369 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.308+0000 7fae922ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:54.369 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.368+0000 7fae922ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:01:54.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:54.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-16T19:01:54.418 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5016e5a640 1 Processor -- start 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5016e5a640 1 -- start start 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5016e5a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5016e5a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f501007be30 con 0x7f5010107d60 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5014bf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5014bf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43700/0 (socket says 172.21.15.62:43700) 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.570+0000 7f5014bf6640 1 -- 172.21.15.62:0/3217140821 learned_addr learned my addr 172.21.15.62:0/3217140821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.571+0000 7f5014bf6640 1 -- 172.21.15.62:0/3217140821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501010a690 con 0x7f5010107d60 2024-08-16T19:01:54.419 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.571+0000 7f5014bf6640 1 --2- 172.21.15.62:0/3217140821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5004009820 tx=0x7f500402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bff403215e24ce server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f500f7fe640 1 -- 172.21.15.62:0/3217140821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5004037070 con 0x7f5010107d60 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f500f7fe640 1 -- 172.21.15.62:0/3217140821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5004032920 con 0x7f5010107d60 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f500f7fe640 1 -- 172.21.15.62:0/3217140821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5004030810 con 0x7f5010107d60 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 -- 172.21.15.62:0/3217140821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 msgr2=0x7f501010a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 --2- 172.21.15.62:0/3217140821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5004009820 tx=0x7f500402a270 comp rx=0 tx=0).stop 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 -- 172.21.15.62:0/3217140821 shutdown_connections 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 --2- 172.21.15.62:0/3217140821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f501010a150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 -- 172.21.15.62:0/3217140821 >> 172.21.15.62:0/3217140821 conn(0x7f5010101a00 msgr2=0x7f5010103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 -- 172.21.15.62:0/3217140821 shutdown_connections 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.572+0000 7f5016e5a640 1 -- 172.21.15.62:0/3217140821 wait complete. 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5016e5a640 1 Processor -- start 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5016e5a640 1 -- start start 2024-08-16T19:01:54.420 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5016e5a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5016e5a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f501010b770 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5014bf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5014bf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43714/0 (socket says 172.21.15.62:43714) 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.573+0000 7f5014bf6640 1 -- 172.21.15.62:0/2668067246 learned_addr learned my addr 172.21.15.62:0/2668067246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f5014bf6640 1 -- 172.21.15.62:0/2668067246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5010107b20 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f5014bf6640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5004030380 tx=0x7f5004032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5004037040 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5004040070 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5010105d10 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f500403a7b0 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.574+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5010106250 con 0x7f5010107d60 2024-08-16T19:01:54.421 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.575+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f500403aa30 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.575+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50101081c0 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.575+0000 7f500dffb640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 0x7f500003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.576+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f500406d6a0 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.576+0000 7f500ffff640 1 -- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 msgr2=0x7f500003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.576+0000 7f500ffff640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 0x7f500003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.580+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5004044c70 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.777+0000 7f500ffff640 1 -- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 msgr2=0x7f500003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.777+0000 7f500ffff640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 0x7f500003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.806+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5010106b50 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.806+0000 7f500dffb640 1 -- 172.21.15.62:0/2668067246 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5004041270 con 0x7f5010107d60 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 msgr2=0x7f500003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:01:54.422 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 0x7f500003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 msgr2=0x7f50101075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5004030380 tx=0x7f5004032670 comp rx=0 tx=0).stop 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 shutdown_connections 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f5000038fc0 0x7f500003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 --2- 172.21.15.62:0/2668067246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5010107d60 0x7f50101075e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 >> 172.21.15.62:0/2668067246 conn(0x7f5010101a00 msgr2=0x7f50101025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 shutdown_connections 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:53.808+0000 7f5016e5a640 1 -- 172.21.15.62:0/2668067246 wait complete. 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for the mgr to restart... 2024-08-16T19:01:54.423 INFO:teuthology.orchestra.run.smithi062.stdout:Waiting for mgr epoch 9... 2024-08-16T19:01:54.640 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2668067246' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-16T19:01:54.641 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.602+0000 7fae922ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:01:54.894 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.672+0000 7fae922ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:01:54.894 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.741+0000 7fae922ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:01:54.894 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:54.893+0000 7fae922ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:01:55.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:55.336+0000 7fae922ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:01:55.892 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:55.666+0000 7fae922ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:01:55.892 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:55.735+0000 7fae922ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:01:55.892 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:55.891+0000 7fae922ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:01:56.476 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.217+0000 7fae922ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:01:56.476 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.281+0000 7fae922ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:01:56.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.475+0000 7fae922ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:01:56.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.630+0000 7fae922ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:01:56.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.697+0000 7fae922ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:01:57.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.783+0000 7fae922ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:01:57.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:56.845+0000 7fae922ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:01:57.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:57.405+0000 7fae922ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:01:57.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:01:57.571+0000 7fae922ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: Active manager daemon x restarted 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: Activating manager daemon x 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: osdmap e3: 0 total, 0 up, 0 in 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: mgrmap e10: x(active, starting, since 0.00315408s) 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: Manager daemon x is now available 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:01:57.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:01:57.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout { 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout } 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f184188b640 1 Processor -- start 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f184188b640 1 -- start start 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f184188b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:59.256 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f184188b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f183c106c80 con 0x7f183c1062b0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f183affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f183affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43730/0 (socket says 172.21.15.62:43730) 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.990+0000 7f183affd640 1 -- 172.21.15.62:0/3075202465 learned_addr learned my addr 172.21.15.62:0/3075202465 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.991+0000 7f183affd640 1 -- 172.21.15.62:0/3075202465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1074b0 con 0x7f183c1062b0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.991+0000 7f183affd640 1 --2- 172.21.15.62:0/3075202465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f182c009820 tx=0x7f182c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89c810733479159 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f183a7fc640 1 -- 172.21.15.62:0/3075202465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c037070 con 0x7f183c1062b0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f183a7fc640 1 -- 172.21.15.62:0/3075202465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182c032920 con 0x7f183c1062b0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f183a7fc640 1 -- 172.21.15.62:0/3075202465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c030810 con 0x7f183c1062b0 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 -- 172.21.15.62:0/3075202465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 msgr2=0x7f183c1066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 --2- 172.21.15.62:0/3075202465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f182c009820 tx=0x7f182c02a270 comp rx=0 tx=0).stop 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 -- 172.21.15.62:0/3075202465 shutdown_connections 2024-08-16T19:01:59.257 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 --2- 172.21.15.62:0/3075202465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c1066b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 -- 172.21.15.62:0/3075202465 >> 172.21.15.62:0/3075202465 conn(0x7f183c101b00 msgr2=0x7f183c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 -- 172.21.15.62:0/3075202465 shutdown_connections 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.992+0000 7f184188b640 1 -- 172.21.15.62:0/3075202465 wait complete. 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f184188b640 1 Processor -- start 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f184188b640 1 -- start start 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f184188b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f184188b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f183c10a860 con 0x7f183c1062b0 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f183affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f183affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43732/0 (socket says 172.21.15.62:43732) 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.993+0000 7f183affd640 1 -- 172.21.15.62:0/2773690392 learned_addr learned my addr 172.21.15.62:0/2773690392 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f183affd640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1919b0 con 0x7f183c1062b0 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f183affd640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f182c030560 tx=0x7f182c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:59.258 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c037040 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f182c040070 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c18fba0 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c03a740 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.994+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1900e0 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f182c03bb90 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1840889640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f182803bbf0 con 0x7f1828039020 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f182c06c580 con 0x7f183c1062b0 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1833fff640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 msgr2=0x7f182803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:54.995+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:55.196+0000 7f1833fff640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 msgr2=0x7f182803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:55.196+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:01:59.259 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:55.597+0000 7f1833fff640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 msgr2=0x7f182803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:55.597+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:56.398+0000 7f1833fff640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 msgr2=0x7f182803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/3198450894 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:56.398+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:57.574+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f182c06b030 con 0x7f183c1062b0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:57.575+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 msgr2=0x7f182803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:57.575+0000 7f1840889640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.577+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f182c03b420 con 0x7f183c1062b0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.577+0000 7f1840889640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 0x7f182803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.577+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f182c03de70 con 0x7f182803cae0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.577+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 0x7f182803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.578+0000 7f1833fff640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 0x7f182803eed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1824003980 tx=0x7f18240092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.579+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f182c03de70 con 0x7f182803cae0 2024-08-16T19:01:59.260 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f183c106730 con 0x7f182803cae0 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f1840889640 1 -- 172.21.15.62:0/2773690392 <== mgr.14150 v2:172.21.15.62:6800/727495888 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f183c106730 con 0x7f182803cae0 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 msgr2=0x7f182803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 0x7f182803eed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1824003980 tx=0x7f18240092b0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 msgr2=0x7f183c191470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f182c030560 tx=0x7f182c032670 comp rx=0 tx=0).stop 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 shutdown_connections 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f182803cae0 0x7f182803eed0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:6800/3198450894,v1:172.21.15.62:6801/3198450894] conn(0x7f1828039020 0x7f182803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 --2- 172.21.15.62:0/2773690392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f183c1062b0 0x7f183c191470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 >> 172.21.15.62:0/2773690392 conn(0x7f183c101b00 msgr2=0x7f183c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 shutdown_connections 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:58.586+0000 7f184188b640 1 -- 172.21.15.62:0/2773690392 wait complete. 2024-08-16T19:01:59.261 INFO:teuthology.orchestra.run.smithi062.stdout:mgr epoch 9 is available 2024-08-16T19:01:59.262 INFO:teuthology.orchestra.run.smithi062.stdout:Generating a dashboard self-signed certificate... 2024-08-16T19:01:59.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:59 smithi062 ceph-mon[32993]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-16T19:01:59.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:59 smithi062 ceph-mon[32993]: mgrmap e11: x(active, since 1.00636s) 2024-08-16T19:01:59.818 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:01:59 smithi062 ceph-mon[32993]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:02:00.089+0000 7fadf7dea640 -1 log_channel(cephadm) log [ERR] : [16/Aug/2024:19:02:00] ENGINE Error in HTTPServer.serve 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Traceback (most recent call last): 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._connections.run(self.expiration_interval) 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._run(expiration_interval) 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:02:00.497 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s = self.context.wrap_socket( 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: return self.sslsocket_class._create( 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.do_handshake() 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._sslobj.do_handshake() 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:02:00.498 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:02:00.553 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-16T19:02:00.553 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.611+0000 7f627c2cf640 1 Processor -- start 2024-08-16T19:02:00.553 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627c2cf640 1 -- start start 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627c2cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627c2cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f627407bdf0 con 0x7f6274107da0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627a06b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627a06b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43790/0 (socket says 172.21.15.62:43790) 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627a06b640 1 -- 172.21.15.62:0/1593046438 learned_addr learned my addr 172.21.15.62:0/1593046438 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.612+0000 7f627a06b640 1 -- 172.21.15.62:0/1593046438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627410a6d0 con 0x7f6274107da0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.613+0000 7f627a06b640 1 --2- 172.21.15.62:0/1593046438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6270009820 tx=0x7f627002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3627a7c2724fc092 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.613+0000 7f6279069640 1 -- 172.21.15.62:0/1593046438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6270037070 con 0x7f6274107da0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.613+0000 7f6279069640 1 -- 172.21.15.62:0/1593046438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270032920 con 0x7f6274107da0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.613+0000 7f6279069640 1 -- 172.21.15.62:0/1593046438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6270030810 con 0x7f6274107da0 2024-08-16T19:02:00.554 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 -- 172.21.15.62:0/1593046438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 msgr2=0x7f627410a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1593046438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6270009820 tx=0x7f627002a270 comp rx=0 tx=0).stop 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 -- 172.21.15.62:0/1593046438 shutdown_connections 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1593046438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627410a190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 -- 172.21.15.62:0/1593046438 >> 172.21.15.62:0/1593046438 conn(0x7f6274101a00 msgr2=0x7f6274103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 -- 172.21.15.62:0/1593046438 shutdown_connections 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.614+0000 7f627c2cf640 1 -- 172.21.15.62:0/1593046438 wait complete. 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627c2cf640 1 Processor -- start 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627c2cf640 1 -- start start 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627c2cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627c2cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f627410b750 con 0x7f6274107da0 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627a06b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627a06b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43806/0 (socket says 172.21.15.62:43806) 2024-08-16T19:02:00.555 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627a06b640 1 -- 172.21.15.62:0/1075001981 learned_addr learned my addr 172.21.15.62:0/1075001981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627a06b640 1 -- 172.21.15.62:0/1075001981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627419da50 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.615+0000 7f627a06b640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62700097f0 tx=0x7f62700327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.616+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6270037040 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.616+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627419dce0 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.616+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6270030cc0 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.616+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62741a09d0 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.616+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f627003a6b0 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.617+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f627003a870 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.617+0000 7f626b7fe640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 0x7f626003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.617+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f627006c620 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.617+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62580052c0 con 0x7f6274107da0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.617+0000 7f627986a640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 0x7f626003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:00.556 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.618+0000 7f627986a640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 0x7f626003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62640098c0 tx=0x7f6264006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.622+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f62700371e0 con 0x7f6274107da0 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.817+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6258000bf0 con 0x7f6260039020 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.935+0000 7f626b7fe640 1 -- 172.21.15.62:0/1075001981 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6258000bf0 con 0x7f6260039020 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 msgr2=0x7f626003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 0x7f626003b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f62640098c0 tx=0x7f6264006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 msgr2=0x7f627419d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f62700097f0 tx=0x7f62700327b0 comp rx=0 tx=0).stop 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 shutdown_connections 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6260039020 0x7f626003b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 --2- 172.21.15.62:0/1075001981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6274107da0 0x7f627419d510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 >> 172.21.15.62:0/1075001981 conn(0x7f6274101a00 msgr2=0x7f6274102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 shutdown_connections 2024-08-16T19:02:00.557 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:01:59.937+0000 7f627c2cf640 1 -- 172.21.15.62:0/1075001981 wait complete. 2024-08-16T19:02:00.558 INFO:teuthology.orchestra.run.smithi062.stdout:Creating initial admin user... 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: mgrmap e12: x(active, since 2s) 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:59] ENGINE Bus STARTING 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: [16/Aug/2024:19:01:59] ENGINE Serving on http://172.21.15.62:8765 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: [16/Aug/2024:19:02:00] ENGINE Serving on https://172.21.15.62:7150 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: [16/Aug/2024:19:02:00] ENGINE Bus STARTED 2024-08-16T19:02:00.839 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:00 smithi062 ceph-mon[32993]: [16/Aug/2024:19:02:00] ENGINE Error in HTTPServer.serve 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-08-16T19:02:00.840 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:02:00.841 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-08-16T19:02:00.841 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:02:00.841 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-08-16T19:02:00.841 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:02:02.056 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:02.057 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:01 smithi062 ceph-mon[32993]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:02.057 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:02.651 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1HQJcLChsY2QAH6hGyhb/OXeNM9EdQTGl.2nxkNhdgSuyuZKUjGkm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1723834921, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-16T19:02:02.651 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.936+0000 7f8890e71640 1 Processor -- start 2024-08-16T19:02:02.651 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.936+0000 7f8890e71640 1 -- start start 2024-08-16T19:02:02.651 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f8890e71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f8890e71640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f888c106bc0 con 0x7f888c1061f0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f888a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f888a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47192/0 (socket says 172.21.15.62:47192) 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f888a59c640 1 -- 172.21.15.62:0/3376326986 learned_addr learned my addr 172.21.15.62:0/3376326986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.937+0000 7f888a59c640 1 -- 172.21.15.62:0/3376326986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c1073f0 con 0x7f888c1061f0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.938+0000 7f888a59c640 1 --2- 172.21.15.62:0/3376326986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f887c00bac0 tx=0x7f887c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257924bdd2de4e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.938+0000 7f888959a640 1 -- 172.21.15.62:0/3376326986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f887c0326d0 con 0x7f888c1061f0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.938+0000 7f888959a640 1 -- 172.21.15.62:0/3376326986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887c038070 con 0x7f888c1061f0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.938+0000 7f888959a640 1 -- 172.21.15.62:0/3376326986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f887c033d00 con 0x7f888c1061f0 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 -- 172.21.15.62:0/3376326986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 msgr2=0x7f888c1065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 --2- 172.21.15.62:0/3376326986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f887c00bac0 tx=0x7f887c02c920 comp rx=0 tx=0).stop 2024-08-16T19:02:02.652 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 -- 172.21.15.62:0/3376326986 shutdown_connections 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 --2- 172.21.15.62:0/3376326986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c1065f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 -- 172.21.15.62:0/3376326986 >> 172.21.15.62:0/3376326986 conn(0x7f888c101a00 msgr2=0x7f888c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 -- 172.21.15.62:0/3376326986 shutdown_connections 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.939+0000 7f8890e71640 1 -- 172.21.15.62:0/3376326986 wait complete. 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f8890e71640 1 Processor -- start 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f8890e71640 1 -- start start 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f8890e71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f8890e71640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f888c07b0d0 con 0x7f888c1061f0 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f888a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f888a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47196/0 (socket says 172.21.15.62:47196) 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f888a59c640 1 -- 172.21.15.62:0/101947198 learned_addr learned my addr 172.21.15.62:0/101947198 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f888a59c640 1 -- 172.21.15.62:0/101947198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c19dab0 con 0x7f888c1061f0 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.940+0000 7f888a59c640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f887c00b2c0 tx=0x7f887c032ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.941+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f887c042070 con 0x7f888c1061f0 2024-08-16T19:02:02.653 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.941+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c19dd40 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.941+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f887c03e920 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.941+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f887c03c9a0 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.941+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c1a0a30 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.942+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f887c045070 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.942+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888c1065f0 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.942+0000 7f887b7fe640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 0x7f887403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.942+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f887c030050 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.942+0000 7f8889d9b640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 0x7f887403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.943+0000 7f8889d9b640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 0x7f887403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88800098c0 tx=0x7f8880006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:00.947+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f887c04c200 con 0x7f888c1061f0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.153+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f888c190f10 con 0x7f8874038fc0 2024-08-16T19:02:02.654 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.403+0000 7f887b7fe640 1 -- 172.21.15.62:0/101947198 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f888c190f10 con 0x7f8874038fc0 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 msgr2=0x7f887403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 0x7f887403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88800098c0 tx=0x7f8880006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 msgr2=0x7f888c19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f887c00b2c0 tx=0x7f887c032ad0 comp rx=0 tx=0).stop 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 shutdown_connections 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8874038fc0 0x7f887403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 --2- 172.21.15.62:0/101947198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f888c1061f0 0x7f888c19d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.405+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 >> 172.21.15.62:0/101947198 conn(0x7f888c101a00 msgr2=0x7f888c18f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.406+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 shutdown_connections 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:01.406+0000 7f8890e71640 1 -- 172.21.15.62:0/101947198 wait complete. 2024-08-16T19:02:02.655 INFO:teuthology.orchestra.run.smithi062.stdout:Fetching dashboard port number... 2024-08-16T19:02:03.674 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:03 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2743862249' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-16T19:02:03.749 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stdout 8443 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe9223d3640 1 Processor -- start 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe9223d3640 1 -- start start 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe9223d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe9223d3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c07e0c0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47198/0 (socket says 172.21.15.62:47198) 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.069+0000 7fe91bfff640 1 -- 172.21.15.62:0/2703779331 learned_addr learned my addr 172.21.15.62:0/2703779331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.070+0000 7fe91bfff640 1 -- 172.21.15.62:0/2703779331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c07c7c0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.070+0000 7fe91bfff640 1 --2- 172.21.15.62:0/2703779331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009820 tx=0x7fe90c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0fe38a9c094f4a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe91affd640 1 -- 172.21.15.62:0/2703779331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c037070 con 0x7fe91c07dcc0 2024-08-16T19:02:03.750 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe91affd640 1 -- 172.21.15.62:0/2703779331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c032920 con 0x7fe91c07dcc0 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe91affd640 1 -- 172.21.15.62:0/2703779331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c030810 con 0x7fe91c07dcc0 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 -- 172.21.15.62:0/2703779331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 msgr2=0x7fe91c07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2703779331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009820 tx=0x7fe90c02a270 comp rx=0 tx=0).stop 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 -- 172.21.15.62:0/2703779331 shutdown_connections 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2703779331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c07c280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 -- 172.21.15.62:0/2703779331 >> 172.21.15.62:0/2703779331 conn(0x7fe91c102c50 msgr2=0x7fe91c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 -- 172.21.15.62:0/2703779331 shutdown_connections 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.071+0000 7fe9223d3640 1 -- 172.21.15.62:0/2703779331 wait complete. 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe9223d3640 1 Processor -- start 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe9223d3640 1 -- start start 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe9223d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:03.751 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe9223d3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c10d900 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47200/0 (socket says 172.21.15.62:47200) 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.072+0000 7fe91bfff640 1 -- 172.21.15.62:0/2743862249 learned_addr learned my addr 172.21.15.62:0/2743862249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe91bfff640 1 -- 172.21.15.62:0/2743862249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c19da40 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe91bfff640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe90c030380 tx=0x7fe90c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c037040 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe90c040070 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c19dcd0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c03a7b0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.073+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c1a09c0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.074+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe90c03a9e0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.074+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c07c300 con 0x7fe91c07dcc0 2024-08-16T19:02:03.752 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.074+0000 7fe9197fa640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 0x7fe90403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.074+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe90c06c810 con 0x7fe91c07dcc0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.075+0000 7fe91b7fe640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 0x7fe90403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.075+0000 7fe91b7fe640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 0x7fe90403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9100098c0 tx=0x7fe910006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.079+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe90c0459e0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.253+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe91c1a0fb0 con 0x7fe91c07dcc0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.254+0000 7fe9197fa640 1 -- 172.21.15.62:0/2743862249 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe90c045460 con 0x7fe91c07dcc0 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 msgr2=0x7fe90403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 0x7fe90403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9100098c0 tx=0x7fe910006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 msgr2=0x7fe91c19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe90c030380 tx=0x7fe90c032670 comp rx=0 tx=0).stop 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 shutdown_connections 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe904039020 0x7fe90403b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:03.753 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 --2- 172.21.15.62:0/2743862249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe91c07dcc0 0x7fe91c19d500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:03.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 >> 172.21.15.62:0/2743862249 conn(0x7fe91c102c50 msgr2=0x7fe91c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:03.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 shutdown_connections 2024-08-16T19:02:03.754 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:03.256+0000 7fe9223d3640 1 -- 172.21.15.62:0/2743862249 wait complete. 2024-08-16T19:02:03.757 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-16T19:02:03.757 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout disabled 2024-08-16T19:02:03.765 INFO:teuthology.orchestra.run.smithi062.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-16T19:02:03.766 INFO:teuthology.orchestra.run.smithi062.stdout:systemctl: stdout inactive 2024-08-16T19:02:03.766 INFO:teuthology.orchestra.run.smithi062.stdout:firewalld.service is not enabled 2024-08-16T19:02:03.766 INFO:teuthology.orchestra.run.smithi062.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-16T19:02:03.767 INFO:teuthology.orchestra.run.smithi062.stdout:Ceph Dashboard is now available at: 2024-08-16T19:02:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: URL: https://smithi062.front.sepia.ceph.com:8443/ 2024-08-16T19:02:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: User: admin 2024-08-16T19:02:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: Password: xieowza6ix 2024-08-16T19:02:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:03.768 INFO:teuthology.orchestra.run.smithi062.stdout:Saving cluster configuration to /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config directory 2024-08-16T19:02:05.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.187+0000 7fa468c0b640 1 Processor -- start 2024-08-16T19:02:05.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.187+0000 7fa468c0b640 1 -- start start 2024-08-16T19:02:05.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.187+0000 7fa468c0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:05.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.187+0000 7fa468c0b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa46407e0a0 con 0x7fa46407dca0 2024-08-16T19:02:05.056 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.188+0000 7fa46259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.188+0000 7fa46259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47210/0 (socket says 172.21.15.62:47210) 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.188+0000 7fa46259c640 1 -- 172.21.15.62:0/3430769047 learned_addr learned my addr 172.21.15.62:0/3430769047 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.188+0000 7fa46259c640 1 -- 172.21.15.62:0/3430769047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46407c7a0 con 0x7fa46407dca0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa46259c640 1 --2- 172.21.15.62:0/3430769047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa454009e80 tx=0x7fa45402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5387ae6e137c1213 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa46159a640 1 -- 172.21.15.62:0/3430769047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa454036070 con 0x7fa46407dca0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa46159a640 1 -- 172.21.15.62:0/3430769047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4540318e0 con 0x7fa46407dca0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa46159a640 1 -- 172.21.15.62:0/3430769047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa454007a00 con 0x7fa46407dca0 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa468c0b640 1 -- 172.21.15.62:0/3430769047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 msgr2=0x7fa46407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.189+0000 7fa468c0b640 1 --2- 172.21.15.62:0/3430769047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa454009e80 tx=0x7fa45402a7a0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 -- 172.21.15.62:0/3430769047 shutdown_connections 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 --2- 172.21.15.62:0/3430769047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46407c260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 -- 172.21.15.62:0/3430769047 >> 172.21.15.62:0/3430769047 conn(0x7fa464102c50 msgr2=0x7fa464105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 -- 172.21.15.62:0/3430769047 shutdown_connections 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 -- 172.21.15.62:0/3430769047 wait complete. 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 Processor -- start 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.190+0000 7fa468c0b640 1 -- start start 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa468c0b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa468c0b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa46410d8d0 con 0x7fa46407dca0 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa46259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa46259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47216/0 (socket says 172.21.15.62:47216) 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa46259c640 1 -- 172.21.15.62:0/2293336931 learned_addr learned my addr 172.21.15.62:0/2293336931 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa46259c640 1 -- 172.21.15.62:0/2293336931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46419da50 con 0x7fa46407dca0 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa46259c640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa45402acc0 tx=0x7fa454031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa454036050 con 0x7fa46407dca0 2024-08-16T19:02:05.058 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa454037040 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.191+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa45403d8e0 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.192+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46419dce0 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.192+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4641a09d0 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.192+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa454044020 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.193+0000 7fa45b7fe640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 0x7fa45003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.193+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46407c2e0 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.193+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa454042020 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.193+0000 7fa461d9b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 0x7fa45003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.193+0000 7fa461d9b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 0x7fa45003b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0098c0 tx=0x7fa44c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.197+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa45404b020 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.307+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa454037ce0 con 0x7fa46407dca0 2024-08-16T19:02:05.059 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.471+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa4641a0fc0 con 0x7fa46407dca0 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.474+0000 7fa45b7fe640 1 -- 172.21.15.62:0/2293336931 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa45402e260 con 0x7fa46407dca0 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 msgr2=0x7fa45003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 0x7fa45003b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0098c0 tx=0x7fa44c006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 msgr2=0x7fa46419d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa45402acc0 tx=0x7fa454031cc0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 shutdown_connections 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa450039020 0x7fa45003b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 --2- 172.21.15.62:0/2293336931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa46407dca0 0x7fa46419d510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 >> 172.21.15.62:0/2293336931 conn(0x7fa464102c50 msgr2=0x7fa4641037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 shutdown_connections 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:/usr/bin/ceph: stderr 2024-08-16T19:02:04.476+0000 7fa468c0b640 1 -- 172.21.15.62:0/2293336931 wait complete. 2024-08-16T19:02:05.060 INFO:teuthology.orchestra.run.smithi062.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout:Or, if you are only running a single cluster on this host: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: ceph telemetry on 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout:For more information see: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-16T19:02:05.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:05.062 INFO:teuthology.orchestra.run.smithi062.stdout:Bootstrap complete. 2024-08-16T19:02:05.098 INFO:tasks.cephadm:Fetching config... 2024-08-16T19:02:05.098 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:05.098 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-16T19:02:05.115 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-16T19:02:05.115 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:05.115 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-16T19:02:05.171 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-16T19:02:05.171 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:05.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-16T19:02:05.237 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-16T19:02:05.237 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:05.237 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-16T19:02:05.293 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-16T19:02:05.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-16T19:02:05.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:05 smithi062 ceph-mon[32993]: mgrmap e13: x(active, since 6s) 2024-08-16T19:02:05.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:05 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2293336931' entity='client.admin' 2024-08-16T19:02:05.392 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:05.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-16T19:02:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:05.457 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-16T19:02:05.498 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJPZw0QZKOFDMqVVDJTadOEB5oWDsKYudWqp0r1tg0Cu1nlQltO7nbETq0Tg+NC4Uz9tQSg1mtaFmYhQkm0b0QnR5jvsCz1dgWplYJuroez/N0fZ8Oeqk0QEoYMEKkGd4ijwOIGYhLwzv6HaGGot1iKGE3sLrExrtFHlhfqS7KK04G28pXRG+5HWDnS7xQvCMMdKqTZrngRb9nIqn2qK8gD+ypis5hN2O1sge7q0UPTnY6IuzDbG/0C4tfut+IX94qKWTiUfTQXiQfM6G1PfF1nIzb23E6H1jvmRCpKTtfsJFFwY7joxfJUh3EIFwSZu2yp6l+AeolIeTYalH9NKVZ1J/TkYHIpfvT22LWeTGgKsHxQT2XOl2fK56Kfj4jQqSimsPsxp7XEL08cpgHWZRYlLhuL/PdvUfCZnUDpi8LYevFAndhZc/fj7Op65KuiZqrQb+KWBPHkT6eIDQRdzdDyOY6q6j0ivDgo4q6MnfPVJej4+HJ6NtjIT9DgGv/ztk= ceph-bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:05.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-16T19:02:05.773 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 -- 172.21.15.62:0/3015281917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 msgr2=0x7fa674100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 --2- 172.21.15.62:0/3015281917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674100670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa664009880 tx=0x7fa66402a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 -- 172.21.15.62:0/3015281917 shutdown_connections 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 --2- 172.21.15.62:0/3015281917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674100670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 -- 172.21.15.62:0/3015281917 >> 172.21.15.62:0/3015281917 conn(0x7fa6740fba00 msgr2=0x7fa6740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 -- 172.21.15.62:0/3015281917 shutdown_connections 2024-08-16T19:02:06.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.259+0000 7fa6790cf640 1 -- 172.21.15.62:0/3015281917 wait complete. 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa6790cf640 1 Processor -- start 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa6790cf640 1 -- start start 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa6790cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa6790cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa674106320 con 0x7fa674100270 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47242/0 (socket says 172.21.15.62:47242) 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa672d9d640 1 -- 172.21.15.62:0/2345017568 learned_addr learned my addr 172.21.15.62:0/2345017568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.260+0000 7fa672d9d640 1 -- 172.21.15.62:0/2345017568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa674196fd0 con 0x7fa674100270 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa672d9d640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa664004f60 tx=0x7fa6640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa66403f070 con 0x7fa674100270 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664004d40 con 0x7fa674100270 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa674197260 con 0x7fa674100270 2024-08-16T19:02:06.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa664035030 con 0x7fa674100270 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.261+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa674199f50 con 0x7fa674100270 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa664042070 con 0x7fa674100270 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa65bfff640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 0x7fa65c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa66406dd90 con 0x7fa674100270 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa674106aa0 con 0x7fa674100270 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa67259c640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 0x7fa65c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.262+0000 7fa67259c640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 0x7fa65c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6680098c0 tx=0x7fa668006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:06.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.267+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa664042360 con 0x7fa674100270 2024-08-16T19:02:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.446+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa67402d5e0 con 0x7fa674100270 2024-08-16T19:02:06.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.448+0000 7fa65bfff640 1 -- 172.21.15.62:0/2345017568 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa66402f300 con 0x7fa674100270 2024-08-16T19:02:06.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 msgr2=0x7fa65c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:06.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 0x7fa65c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa6680098c0 tx=0x7fa668006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 msgr2=0x7fa674196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa664004f60 tx=0x7fa6640048f0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 shutdown_connections 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa65c039060 0x7fa65c03b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 --2- 172.21.15.62:0/2345017568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa674100270 0x7fa674196a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 >> 172.21.15.62:0/2345017568 conn(0x7fa6740fba00 msgr2=0x7fa674069840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 shutdown_connections 2024-08-16T19:02:06.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:06.451+0000 7fa6790cf640 1 -- 172.21.15.62:0/2345017568 wait complete. 2024-08-16T19:02:07.424 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-16T19:02:07.425 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-16T19:02:07.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:07 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2345017568' entity='client.admin' 2024-08-16T19:02:07.873 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 -- 172.21.15.62:0/934365887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 msgr2=0x7ff98c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 --2- 172.21.15.62:0/934365887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c1041f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009880 tx=0x7ff97c02a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 -- 172.21.15.62:0/934365887 shutdown_connections 2024-08-16T19:02:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 --2- 172.21.15.62:0/934365887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c1041f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 -- 172.21.15.62:0/934365887 >> 172.21.15.62:0/934365887 conn(0x7ff98c0fba00 msgr2=0x7ff98c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 -- 172.21.15.62:0/934365887 shutdown_connections 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.340+0000 7ff991709640 1 -- 172.21.15.62:0/934365887 wait complete. 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff991709640 1 Processor -- start 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff991709640 1 -- start start 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff991709640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff991709640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff98c1073a0 con 0x7ff98c101e00 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff98affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff98affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47266/0 (socket says 172.21.15.62:47266) 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff98affd640 1 -- 172.21.15.62:0/1156698668 learned_addr learned my addr 172.21.15.62:0/1156698668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff98affd640 1 -- 172.21.15.62:0/1156698668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98c196fb0 con 0x7ff98c101e00 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.341+0000 7ff98affd640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff97c004f60 tx=0x7ff97c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.342+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff97c03f070 con 0x7ff98c101e00 2024-08-16T19:02:08.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.342+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98c1972a0 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.342+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97c003ad0 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.342+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff97c035030 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.342+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98c199f30 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.343+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff97c042070 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.343+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff98c107b20 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.343+0000 7ff988ff9640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 0x7ff97803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.343+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff97c06cd40 con 0x7ff98c101e00 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.343+0000 7ff983fff640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 0x7ff97803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.344+0000 7ff983fff640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 0x7ff97803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9740098c0 tx=0x7ff974006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:08.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.350+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff97c039350 con 0x7ff98c101e00 2024-08-16T19:02:08.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.534+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff98c02d7c0 con 0x7ff9780390c0 2024-08-16T19:02:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.538+0000 7ff988ff9640 1 -- 172.21.15.62:0/1156698668 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff98c02d7c0 con 0x7ff9780390c0 2024-08-16T19:02:08.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 msgr2=0x7ff97803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 0x7ff97803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9740098c0 tx=0x7ff974006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 msgr2=0x7ff98c196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff97c004f60 tx=0x7ff97c0048f0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 shutdown_connections 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff9780390c0 0x7ff97803b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 --2- 172.21.15.62:0/1156698668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff98c101e00 0x7ff98c196a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 >> 172.21.15.62:0/1156698668 conn(0x7ff98c0fba00 msgr2=0x7ff98c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 shutdown_connections 2024-08-16T19:02:08.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:08.540+0000 7ff991709640 1 -- 172.21.15.62:0/1156698668 wait complete. 2024-08-16T19:02:09.198 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-08-16T19:02:09.198 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:02:09.199 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-08-16T19:02:09.216 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:02:09.217 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:09.274 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-08-16T19:02:09.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch host add smithi155 2024-08-16T19:02:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:09.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:09 smithi062 ceph-mon[32993]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:09.795 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:09 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 -- 172.21.15.62:0/1906083182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 msgr2=0x7fbc901041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:10.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 --2- 172.21.15.62:0/1906083182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc901041f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009880 tx=0x7fbc8c02a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:10.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 -- 172.21.15.62:0/1906083182 shutdown_connections 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 --2- 172.21.15.62:0/1906083182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc901041f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 -- 172.21.15.62:0/1906083182 >> 172.21.15.62:0/1906083182 conn(0x7fbc900fba00 msgr2=0x7fbc900fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 -- 172.21.15.62:0/1906083182 shutdown_connections 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.139+0000 7fbc982ff640 1 -- 172.21.15.62:0/1906083182 wait complete. 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc982ff640 1 Processor -- start 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc982ff640 1 -- start start 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc982ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc982ff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901073a0 con 0x7fbc90101e00 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc9609b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc9609b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47286/0 (socket says 172.21.15.62:47286) 2024-08-16T19:02:10.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.140+0000 7fbc9609b640 1 -- 172.21.15.62:0/3385308047 learned_addr learned my addr 172.21.15.62:0/3385308047 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc9609b640 1 -- 172.21.15.62:0/3385308047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90199170 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc9609b640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c004f60 tx=0x7fbc8c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc8c03f070 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc90199460 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c003ad0 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc8c035030 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.141+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9019c0f0 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.142+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbc8c042070 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.142+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc90107b20 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.142+0000 7fbc877fe640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 0x7fbc7c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.142+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc8c06cd90 con 0x7fbc90101e00 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.142+0000 7fbc9589a640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 0x7fbc7c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:10.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.143+0000 7fbc9589a640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 0x7fbc7c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc800098c0 tx=0x7fbc80006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:10.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.147+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbc8c039350 con 0x7fbc90101e00 2024-08-16T19:02:10.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:10.329+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) -- 0x7fbc9002d7d0 con 0x7fbc7c039060 2024-08-16T19:02:10.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:10.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:10.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:10.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:10.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:11 smithi062 ceph-mon[32993]: Deploying cephadm binary to smithi155 2024-08-16T19:02:14.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.214+0000 7fbc877fe640 1 -- 172.21.15.62:0/3385308047 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbc9002d7d0 con 0x7fbc7c039060 2024-08-16T19:02:14.215 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-08-16T19:02:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 msgr2=0x7fbc7c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 0x7fbc7c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc800098c0 tx=0x7fbc80006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 msgr2=0x7fbc90198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c004f60 tx=0x7fbc8c0048f0 comp rx=0 tx=0).stop 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 shutdown_connections 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbc7c039060 0x7fbc7c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 --2- 172.21.15.62:0/3385308047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc90101e00 0x7fbc90198c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 >> 172.21.15.62:0/3385308047 conn(0x7fbc900fba00 msgr2=0x7fbc900fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 shutdown_connections 2024-08-16T19:02:14.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:14.217+0000 7fbc982ff640 1 -- 172.21.15.62:0/3385308047 wait complete. 2024-08-16T19:02:14.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-16T19:02:15.075 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:15.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:15.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:15 smithi062 ceph-mon[32993]: Added host smithi155 2024-08-16T19:02:15.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:15.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:15.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.515+0000 7f8091c7f640 1 -- 172.21.15.62:0/1702638604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 msgr2=0x7f808c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.515+0000 7f8091c7f640 1 --2- 172.21.15.62:0/1702638604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1028c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f807c009880 tx=0x7f807c02a670 comp rx=0 tx=0).stop 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 -- 172.21.15.62:0/1702638604 shutdown_connections 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 --2- 172.21.15.62:0/1702638604 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1028c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 -- 172.21.15.62:0/1702638604 >> 172.21.15.62:0/1702638604 conn(0x7f808c0fbee0 msgr2=0x7f808c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 -- 172.21.15.62:0/1702638604 shutdown_connections 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 -- 172.21.15.62:0/1702638604 wait complete. 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 Processor -- start 2024-08-16T19:02:15.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.516+0000 7f8091c7f640 1 -- start start 2024-08-16T19:02:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f8091c7f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f8091c7f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10b9e0 con 0x7f808c1024e0 2024-08-16T19:02:15.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f808b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f808b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44202/0 (socket says 172.21.15.62:44202) 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f808b7fe640 1 -- 172.21.15.62:0/3533604998 learned_addr learned my addr 172.21.15.62:0/3533604998 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f808b7fe640 1 -- 172.21.15.62:0/3533604998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c199620 con 0x7f808c1024e0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f808b7fe640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f807c009230 tx=0x7f807c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c03f070 con 0x7f808c1024e0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.518+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1998b0 con 0x7f808c1024e0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.517+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f807c03bd40 con 0x7f808c1024e0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.518+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c035030 con 0x7f808c1024e0 2024-08-16T19:02:15.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.518+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c199d10 con 0x7f808c1024e0 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.518+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f807c042070 con 0x7f808c1024e0 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.519+0000 7f8088ff9640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 0x7f807403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.519+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f807c06cc90 con 0x7f808c1024e0 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.519+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c06a480 con 0x7f808c1024e0 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.519+0000 7f808affd640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 0x7f807403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:15.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.519+0000 7f808affd640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 0x7f807403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80800098c0 tx=0x7f8080006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:15.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.524+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f807c03a890 con 0x7f808c1024e0 2024-08-16T19:02:15.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.705+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f808c19a470 con 0x7f8074039060 2024-08-16T19:02:15.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.706+0000 7f8088ff9640 1 -- 172.21.15.62:0/3533604998 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f808c19a470 con 0x7f8074039060 2024-08-16T19:02:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:15.707 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-08-16T19:02:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 msgr2=0x7f807403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:15.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 0x7f807403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80800098c0 tx=0x7f8080006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 msgr2=0x7f808c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f807c009230 tx=0x7f807c003d80 comp rx=0 tx=0).stop 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 shutdown_connections 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8074039060 0x7f807403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 --2- 172.21.15.62:0/3533604998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f808c1024e0 0x7f808c1990e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 >> 172.21.15.62:0/3533604998 conn(0x7f808c0fbee0 msgr2=0x7f808c1076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 shutdown_connections 2024-08-16T19:02:15.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:15.709+0000 7f8091c7f640 1 -- 172.21.15.62:0/3533604998 wait complete. 2024-08-16T19:02:16.273 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-08-16T19:02:16.273 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:02:16.273 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-08-16T19:02:16.292 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:02:16.292 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:16.349 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-08-16T19:02:16.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch host add smithi161 2024-08-16T19:02:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:16.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:16 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:16.873 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:16 smithi062 ceph-mon[32993]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 -- 172.21.15.62:0/1308715143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 msgr2=0x7ff7a8100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/1308715143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8100800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff790009880 tx=0x7ff79002a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 -- 172.21.15.62:0/1308715143 shutdown_connections 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/1308715143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8100800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 -- 172.21.15.62:0/1308715143 >> 172.21.15.62:0/1308715143 conn(0x7ff7a80fbee0 msgr2=0x7ff7a80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 -- 172.21.15.62:0/1308715143 shutdown_connections 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.955+0000 7ff7aeed2640 1 -- 172.21.15.62:0/1308715143 wait complete. 2024-08-16T19:02:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7aeed2640 1 Processor -- start 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7aeed2640 1 -- start start 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7aeed2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7aeed2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a810a870 con 0x7ff7a8100420 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7acc6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7acc6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44208/0 (socket says 172.21.15.62:44208) 2024-08-16T19:02:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.956+0000 7ff7acc6e640 1 -- 172.21.15.62:0/4226053729 learned_addr learned my addr 172.21.15.62:0/4226053729 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:16.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7acc6e640 1 -- 172.21.15.62:0/4226053729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81962e0 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7acc6e640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff790009230 tx=0x7ff790003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff79003f070 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79003bc80 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a81965d0 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff790035030 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.957+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a819a670 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.958+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff790042070 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.958+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a8101900 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.958+0000 7ff7a5ffb640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 0x7ff79403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.958+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff79006ccc0 con 0x7ff7a8100420 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.959+0000 7ff7a7fff640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 0x7ff79403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.959+0000 7ff7a7fff640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 0x7ff79403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ad30 tx=0x7ff7980093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:16.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:16.963+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff790031570 con 0x7ff7a8100420 2024-08-16T19:02:17.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:17.143+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7ff7a8100880 con 0x7ff794039060 2024-08-16T19:02:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:17 smithi062 ceph-mon[32993]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:19.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:19.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: Deploying cephadm binary to smithi161 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:19.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:20.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:20 smithi062 ceph-mon[32993]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:02:21.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.023+0000 7ff7a5ffb640 1 -- 172.21.15.62:0/4226053729 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff7a8100880 con 0x7ff794039060 2024-08-16T19:02:21.025 INFO:teuthology.orchestra.run.smithi062.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 msgr2=0x7ff79403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 0x7ff79403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff79800ad30 tx=0x7ff7980093f0 comp rx=0 tx=0).stop 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 msgr2=0x7ff7a8195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff790009230 tx=0x7ff790003d40 comp rx=0 tx=0).stop 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 shutdown_connections 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ff794039060 0x7ff79403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 --2- 172.21.15.62:0/4226053729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7a8100420 0x7ff7a8195da0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 >> 172.21.15.62:0/4226053729 conn(0x7ff7a80fbee0 msgr2=0x7ff7a80fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:21.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 shutdown_connections 2024-08-16T19:02:21.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:21.025+0000 7ff7aeed2640 1 -- 172.21.15.62:0/4226053729 wait complete. 2024-08-16T19:02:21.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:21 smithi062 ceph-mon[32993]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:21.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:21.335 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:21.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-16T19:02:21.926 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 -- 172.21.15.62:0/351390400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 msgr2=0x7f6bf8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/351390400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf8101900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009880 tx=0x7f6be002a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 -- 172.21.15.62:0/351390400 shutdown_connections 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/351390400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf8101900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 -- 172.21.15.62:0/351390400 >> 172.21.15.62:0/351390400 conn(0x7f6bf80fcfc0 msgr2=0x7f6bf80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 -- 172.21.15.62:0/351390400 shutdown_connections 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.320+0000 7f6bfeeef640 1 -- 172.21.15.62:0/351390400 wait complete. 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfeeef640 1 Processor -- start 2024-08-16T19:02:22.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfeeef640 1 -- start start 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfeeef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfeeef640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810b970 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfcc8b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfcc8b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42898/0 (socket says 172.21.15.62:42898) 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfcc8b640 1 -- 172.21.15.62:0/3140501896 learned_addr learned my addr 172.21.15.62:0/3140501896 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfcc8b640 1 -- 172.21.15.62:0/3140501896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf81995e0 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.321+0000 7f6bfcc8b640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009230 tx=0x7f6be0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.322+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be003f070 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.322+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be003bc80 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.322+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81998d0 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.322+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be0035030 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.322+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf819d970 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.323+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6be0042070 con 0x7f6bf8101520 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.323+0000 7f6bf5ffb640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 0x7f6be403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.323+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6be006cc60 con 0x7f6bf8101520 2024-08-16T19:02:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.323+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf8102a00 con 0x7f6bf8101520 2024-08-16T19:02:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.323+0000 7f6bf7fff640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 0x7f6be403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.324+0000 7f6bf7fff640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 0x7f6be403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098c0 tx=0x7f6be8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.328+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6be0031dd0 con 0x7f6bf8101520 2024-08-16T19:02:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.510+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bf8101980 con 0x7f6be40390c0 2024-08-16T19:02:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.511+0000 7f6bf5ffb640 1 -- 172.21.15.62:0/3140501896 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f6bf8101980 con 0x7f6be40390c0 2024-08-16T19:02:22.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:[{"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-08-16T19:02:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 msgr2=0x7f6be403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 0x7f6be403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098c0 tx=0x7f6be8006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 msgr2=0x7f6bf81990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009230 tx=0x7f6be0003d40 comp rx=0 tx=0).stop 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 shutdown_connections 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6be40390c0 0x7f6be403b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 --2- 172.21.15.62:0/3140501896 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bf8101520 0x7f6bf81990a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 >> 172.21.15.62:0/3140501896 conn(0x7f6bf80fcfc0 msgr2=0x7f6bf8109420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 shutdown_connections 2024-08-16T19:02:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:22.513+0000 7f6bfeeef640 1 -- 172.21.15.62:0/3140501896 wait complete. 2024-08-16T19:02:22.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:22 smithi062 ceph-mon[32993]: Added host smithi161 2024-08-16T19:02:22.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:22 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:22.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:22 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:23.079 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-16T19:02:23.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-16T19:02:23.339 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:23.694 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:23 smithi062 ceph-mon[32993]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:23.695 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:23 smithi062 ceph-mon[32993]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:02:23.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.775+0000 7f8631199640 1 -- 172.21.15.62:0/3589721753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 msgr2=0x7f862c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:23.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.775+0000 7f8631199640 1 --2- 172.21.15.62:0/3589721753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c101900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8614009c20 tx=0x7f861402aa90 comp rx=0 tx=0).stop 2024-08-16T19:02:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 -- 172.21.15.62:0/3589721753 shutdown_connections 2024-08-16T19:02:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 --2- 172.21.15.62:0/3589721753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c101900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 -- 172.21.15.62:0/3589721753 >> 172.21.15.62:0/3589721753 conn(0x7f862c0fcfc0 msgr2=0x7f862c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 -- 172.21.15.62:0/3589721753 shutdown_connections 2024-08-16T19:02:23.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 -- 172.21.15.62:0/3589721753 wait complete. 2024-08-16T19:02:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.776+0000 7f8631199640 1 Processor -- start 2024-08-16T19:02:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f8631199640 1 -- start start 2024-08-16T19:02:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f8631199640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f8631199640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f862c10b970 con 0x7f862c101520 2024-08-16T19:02:23.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f862ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f862ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42904/0 (socket says 172.21.15.62:42904) 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f862ad9d640 1 -- 172.21.15.62:0/1548855210 learned_addr learned my addr 172.21.15.62:0/1548855210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f862ad9d640 1 -- 172.21.15.62:0/1548855210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c197410 con 0x7f862c101520 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.777+0000 7f862ad9d640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8614005e30 tx=0x7f861402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.778+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8614047070 con 0x7f862c101520 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.778+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c197700 con 0x7f862c101520 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.778+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614003df0 con 0x7f862c101520 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.778+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8614033050 con 0x7f862c101520 2024-08-16T19:02:23.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.778+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c19b7a0 con 0x7f862c101520 2024-08-16T19:02:23.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f861404a070 con 0x7f862c101520 2024-08-16T19:02:23.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c102a00 con 0x7f862c101520 2024-08-16T19:02:23.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f8623fff640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 0x7f861803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:23.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f861403e030 con 0x7f862c101520 2024-08-16T19:02:23.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f862a59c640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 0x7f861803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:23.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.779+0000 7f862a59c640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 0x7f861803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098c0 tx=0x7f861c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:23.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.784+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f86140373b0 con 0x7f862c101520 2024-08-16T19:02:23.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:23.963+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f862c198230 con 0x7f862c101520 2024-08-16T19:02:24.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.378+0000 7f8623fff640 1 -- 172.21.15.62:0/1548855210 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f86140492c0 con 0x7f862c101520 2024-08-16T19:02:24.380 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 msgr2=0x7f861803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 0x7f861803b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098c0 tx=0x7f861c006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 msgr2=0x7f862c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8614005e30 tx=0x7f861402fc10 comp rx=0 tx=0).stop 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 shutdown_connections 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f86180390c0 0x7f861803b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 --2- 172.21.15.62:0/1548855210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f862c101520 0x7f862c196ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 >> 172.21.15.62:0/1548855210 conn(0x7f862c0fcfc0 msgr2=0x7f862c108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:24.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 shutdown_connections 2024-08-16T19:02:24.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:24.380+0000 7f8631199640 1 -- 172.21.15.62:0/1548855210 wait complete. 2024-08-16T19:02:24.681 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:24 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1548855210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-16T19:02:24.948 INFO:tasks.cephadm:Adding mon.a on smithi062 2024-08-16T19:02:24.948 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-08-16T19:02:24.948 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-08-16T19:02:24.948 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi062:172.21.15.62=a;smithi155:172.21.15.155=b;smithi161:172.21.15.161=c' 2024-08-16T19:02:25.193 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-16T19:02:25.241 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-16T19:02:25.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 -- 172.21.15.161:0/1398233220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 msgr2=0x7f4430100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:25.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 --2- 172.21.15.161:0/1398233220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430100670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4420009880 tx=0x7f442002a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 -- 172.21.15.161:0/1398233220 shutdown_connections 2024-08-16T19:02:25.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 --2- 172.21.15.161:0/1398233220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430100670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.726 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 -- 172.21.15.161:0/1398233220 >> 172.21.15.161:0/1398233220 conn(0x7f44300fba00 msgr2=0x7f44300fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:25.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 -- 172.21.15.161:0/1398233220 shutdown_connections 2024-08-16T19:02:25.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.725+0000 7f44369c3640 1 -- 172.21.15.161:0/1398233220 wait complete. 2024-08-16T19:02:25.727 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f44369c3640 1 Processor -- start 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f44369c3640 1 -- start start 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f44369c3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f44369c3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4430106320 con 0x7f4430100270 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f442ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f442ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.161:52334/0 (socket says 172.21.15.161:52334) 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.726+0000 7f442ffff640 1 -- 172.21.15.161:0/2412535251 learned_addr learned my addr 172.21.15.161:0/2412535251 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f442ffff640 1 -- 172.21.15.161:0/2412535251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430196fe0 con 0x7f4430100270 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f442ffff640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4420004f60 tx=0x7f44200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f442003f070 con 0x7f4430100270 2024-08-16T19:02:25.728 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4420004d40 con 0x7f4430100270 2024-08-16T19:02:25.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420035030 con 0x7f4430100270 2024-08-16T19:02:25.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4430197270 con 0x7f4430100270 2024-08-16T19:02:25.729 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.727+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4430199f60 con 0x7f4430100270 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.728+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4420042070 con 0x7f4430100270 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.728+0000 7f442d7fa640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 0x7f441803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.728+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f442006cea0 con 0x7f4430100270 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.728+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4430106aa0 con 0x7f4430100270 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.728+0000 7f442f7fe640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 0x7f441803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:25.730 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.729+0000 7f442f7fe640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 0x7f441803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44240098c0 tx=0x7f4424006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:25.734 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.733+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4420042360 con 0x7f4430100270 2024-08-16T19:02:25.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1548855210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:25.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:25.924 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.924+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi155:172.21.15.155=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) -- 0x7f4430062360 con 0x7f44180390c0 2024-08-16T19:02:25.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.929+0000 7f442d7fa640 1 -- 172.21.15.161:0/2412535251 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4430062360 con 0x7f44180390c0 2024-08-16T19:02:25.930 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 msgr2=0x7f441803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 0x7f441803b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44240098c0 tx=0x7f4424006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 msgr2=0x7f4430196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4420004f60 tx=0x7f44200048f0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 shutdown_connections 2024-08-16T19:02:25.932 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44180390c0 0x7f441803b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 --2- 172.21.15.161:0/2412535251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4430100270 0x7f4430196aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:25.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.931+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 >> 172.21.15.161:0/2412535251 conn(0x7f44300fba00 msgr2=0x7f4430069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:25.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.932+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 shutdown_connections 2024-08-16T19:02:25.933 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:25.932+0000 7f44369c3640 1 -- 172.21.15.161:0/2412535251 wait complete. 2024-08-16T19:02:26.559 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.b.service 2024-08-16T19:02:26.562 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.c.service 2024-08-16T19:02:26.565 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-16T19:02:26.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-16T19:02:26.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:02:26.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:02:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi062:172.21.15.62=a;smithi155:172.21.15.155=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: Saving service mon spec with placement smithi062:172.21.15.62=a;smithi155:172.21.15.155=b;smithi161:172.21.15.161=c;count:3 2024-08-16T19:02:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:27 smithi062 ceph-mon[32993]: Deploying daemon mon.c on smithi161 2024-08-16T19:02:28.569 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:02:29.112 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.109+0000 7f6d1515b640 1 -- 172.21.15.161:0/845504619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 msgr2=0x7f6d10100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.109+0000 7f6d1515b640 1 --2- 172.21.15.161:0/845504619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10100900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009880 tx=0x7f6d0802a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 -- 172.21.15.161:0/845504619 shutdown_connections 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 --2- 172.21.15.161:0/845504619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10100900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 -- 172.21.15.161:0/845504619 >> 172.21.15.161:0/845504619 conn(0x7f6d100fbf80 msgr2=0x7f6d100fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 -- 172.21.15.161:0/845504619 shutdown_connections 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 -- 172.21.15.161:0/845504619 wait complete. 2024-08-16T19:02:29.113 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.110+0000 7f6d1515b640 1 Processor -- start 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d1515b640 1 -- start start 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d1515b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d1515b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010a970 con 0x7f6d10100520 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d0ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.161:52346/0 (socket says 172.21.15.161:52346) 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.111+0000 7f6d0ed9d640 1 -- 172.21.15.161:0/3747842739 learned_addr learned my addr 172.21.15.161:0/3747842739 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:29.114 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d0ed9d640 1 -- 172.21.15.161:0/3747842739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1006af60 con 0x7f6d10100520 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d0ed9d640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009230 tx=0x7f6d08003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d0803f070 con 0x7f6d10100520 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d0803bc80 con 0x7f6d10100520 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d10196190 con 0x7f6d10100520 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d08035030 con 0x7f6d10100520 2024-08-16T19:02:29.115 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.112+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d10196650 con 0x7f6d10100520 2024-08-16T19:02:29.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.113+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6d08042070 con 0x7f6d10100520 2024-08-16T19:02:29.116 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.113+0000 7f6d07fff640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 0x7f6cfc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:29.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.113+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6d0806cc70 con 0x7f6d10100520 2024-08-16T19:02:29.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.113+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d10101a00 con 0x7f6d10100520 2024-08-16T19:02:29.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.113+0000 7f6d0e59c640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 0x7f6cfc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:29.117 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.114+0000 7f6d0e59c640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 0x7f6cfc03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098c0 tx=0x7f6cf8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:29.119 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.119+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d0804c350 con 0x7f6d10100520 2024-08-16T19:02:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.408+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6d1006a750 con 0x7f6d10100520 2024-08-16T19:02:29.409 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.409+0000 7f6d07fff640 1 -- 172.21.15.161:0/3747842739 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6d0804ad50 con 0x7f6d10100520 2024-08-16T19:02:29.410 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:02:29.410 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","modified":"2024-08-16T19:00:59.803011Z","created":"2024-08-16T19:00:59.803011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-16T19:02:29.410 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 msgr2=0x7f6cfc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 0x7f6cfc03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098c0 tx=0x7f6cf8006db0 comp rx=0 tx=0).stop 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 msgr2=0x7f6d10195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009230 tx=0x7f6d08003d40 comp rx=0 tx=0).stop 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 shutdown_connections 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6cfc039060 0x7f6cfc03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 --2- 172.21.15.161:0/3747842739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d10100520 0x7f6d10195c50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:29.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.411+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 >> 172.21.15.161:0/3747842739 conn(0x7f6d100fbf80 msgr2=0x7f6d10107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:29.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.412+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 shutdown_connections 2024-08-16T19:02:29.413 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:29.412+0000 7f6d1515b640 1 -- 172.21.15.161:0/3747842739 wait complete. 2024-08-16T19:02:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:29 smithi062 ceph-mon[32993]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:30.379 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:30 smithi161 ceph-mon[37576]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-16T19:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:30 smithi161 bash[37557]: 171681b2b1a444c98748c56e6380bbc11ca61e67f79a5d7d3eed9f96be1e7552 2024-08-16T19:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:30 smithi161 systemd[1]: Started Ceph mon.c for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:02:30.938 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-16T19:02:30.938 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-16T19:02:31.189 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: mon.a calling monitor election 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: mon.c calling monitor election 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: monmap epoch 2 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: last_changed 2024-08-16T19:02:30.358525+0000 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: min_mon_release 19 (squid) 2024-08-16T19:02:35.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: election_strategy: 1 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: fsmap 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: mgrmap e13: x(active, since 37s) 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: overall HEALTH_OK 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:35.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:02:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: mon.a calling monitor election 2024-08-16T19:02:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: mon.c calling monitor election 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: monmap epoch 2 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: last_changed 2024-08-16T19:02:30.358525+0000 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:02:35.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: min_mon_release 19 (squid) 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: election_strategy: 1 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: fsmap 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: mgrmap e13: x(active, since 37s) 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: overall HEALTH_OK 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:35.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:35.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 -- 172.21.15.161:0/4080319130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa24c003530 msgr2=0x7fa24c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:35.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 --2- 172.21.15.161:0/4080319130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa24c003530 0x7fa24c0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa260102830 tx=0x7fa25c02c490 comp rx=0 tx=0).stop 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 -- 172.21.15.161:0/4080319130 shutdown_connections 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 --2- 172.21.15.161:0/4080319130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa24c003530 0x7fa24c0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 -- 172.21.15.161:0/4080319130 >> 172.21.15.161:0/4080319130 conn(0x7fa2600fd080 msgr2=0x7fa2600ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 -- 172.21.15.161:0/4080319130 shutdown_connections 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.990+0000 7fa267d90640 1 -- 172.21.15.161:0/4080319130 wait complete. 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 Processor -- start 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 -- start start 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 0x7fa260074bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2601094b0 con 0x7fa260074760 2024-08-16T19:02:35.994 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.991+0000 7fa267d90640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa260109330 con 0x7fa24c003530 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44662/0 (socket says 172.21.15.161:44662) 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 -- 172.21.15.161:0/1225244924 learned_addr learned my addr 172.21.15.161:0/1225244924 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa26532b640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 0x7fa260074bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 -- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 msgr2=0x7fa260074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 -- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 msgr2=0x7fa260074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa265b2c640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa26532b640 1 -- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 msgr2=0x7fa260074220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa26532b640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa26532b640 1 -- 172.21.15.161:0/1225244924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa260076110 con 0x7fa260074760 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa26532b640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 0x7fa260074bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b400 tx=0x7fa25000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:35.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.992+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa250011630 con 0x7fa260074760 2024-08-16T19:02:35.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.993+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa250011cb0 con 0x7fa260074760 2024-08-16T19:02:35.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.993+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa250010d80 con 0x7fa260074760 2024-08-16T19:02:35.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.993+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2601a3300 con 0x7fa260074760 2024-08-16T19:02:35.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.993+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2601a3890 con 0x7fa260074760 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.994+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa250018020 con 0x7fa260074760 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.994+0000 7fa256ffd640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 0x7fa24403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.994+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa25004bca0 con 0x7fa260074760 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.994+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa260102ae0 con 0x7fa260074760 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.994+0000 7fa265b2c640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 0x7fa24403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:35.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:35.995+0000 7fa265b2c640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 0x7fa24403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa25c012c10 tx=0x7fa25c0323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:36.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.000+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa25001f020 con 0x7fa260074760 2024-08-16T19:02:36.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.238+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa260190c90 con 0x7fa260074760 2024-08-16T19:02:36.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.239+0000 7fa256ffd640 1 -- 172.21.15.161:0/1225244924 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa250018310 con 0x7fa260074760 2024-08-16T19:02:36.240 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:02:36.240 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","modified":"2024-08-16T19:02:30.358525Z","created":"2024-08-16T19:00:59.803011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-16T19:02:36.240 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 msgr2=0x7fa24403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 0x7fa24403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa25c012c10 tx=0x7fa25c0323a0 comp rx=0 tx=0).stop 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 msgr2=0x7fa260074bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 0x7fa260074bc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b400 tx=0x7fa25000b8c0 comp rx=0 tx=0).stop 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 shutdown_connections 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa244038e50 0x7fa24403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa260074760 0x7fa260074bc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 --2- 172.21.15.161:0/1225244924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa24c003530 0x7fa260074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 >> 172.21.15.161:0/1225244924 conn(0x7fa2600fd080 msgr2=0x7fa2600fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 shutdown_connections 2024-08-16T19:02:36.242 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:36.241+0000 7fa267d90640 1 -- 172.21.15.161:0/1225244924 wait complete. 2024-08-16T19:02:36.525 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:36 smithi161 ceph-mon[37576]: Deploying daemon mon.b on smithi155 2024-08-16T19:02:36.525 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:36 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/1225244924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-16T19:02:36.525 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:36 smithi062 ceph-mon[32993]: Deploying daemon mon.b on smithi155 2024-08-16T19:02:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:36 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/1225244924' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-16T19:02:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:37 smithi062 ceph-mon[32993]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:37.811 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-16T19:02:37.811 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-16T19:02:37.837 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:37 smithi161 ceph-mon[37576]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:38.069 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.480+0000 7f012068b640 1 -- 172.21.15.161:0/1728034479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01181004f0 msgr2=0x7f01181008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.480+0000 7f012068b640 1 --2- 172.21.15.161:0/1728034479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01181004f0 0x7f01181008d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0114009880 tx=0x7f011402a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 -- 172.21.15.161:0/1728034479 shutdown_connections 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 --2- 172.21.15.161:0/1728034479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01181004f0 0x7f01181008d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 -- 172.21.15.161:0/1728034479 >> 172.21.15.161:0/1728034479 conn(0x7f01180fbf50 msgr2=0x7f01180fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 -- 172.21.15.161:0/1728034479 shutdown_connections 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 -- 172.21.15.161:0/1728034479 wait complete. 2024-08-16T19:02:38.486 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 Processor -- start 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.481+0000 7f012068b640 1 -- start start 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f012068b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f012068b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 0x7f011819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f012068b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f011810a940 con 0x7f0118196580 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f012068b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f011810a7c0 con 0x7f01181004f0 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:38.487 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:44670/0 (socket says 172.21.15.161:44670) 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 -- 172.21.15.161:0/2152005000 learned_addr learned my addr 172.21.15.161:0/2152005000 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011dc26640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 0x7f011819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 -- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 msgr2=0x7f0118196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 -- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 msgr2=0x7f0118196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011e427640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:02:38.488 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011dc26640 1 -- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 msgr2=0x7f0118196040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011dc26640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011dc26640 1 -- 172.21.15.161:0/2152005000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0118196ba0 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.482+0000 7f011dc26640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 0x7f011819a8c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f010800b3d0 tx=0x7f010800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0108011630 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819ae00 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0108011cb0 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0108010df0 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f011819b340 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.483+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01181019d0 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.484+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0108018020 con 0x7f0118196580 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.484+0000 7f010f7fe640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 0x7f010403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:38.489 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.484+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f010804bcf0 con 0x7f0118196580 2024-08-16T19:02:38.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.484+0000 7f011e427640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 0x7f010403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:38.490 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.484+0000 7f011e427640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 0x7f010403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01140096c0 tx=0x7f0114005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:38.491 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.489+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f010801f020 con 0x7f0118196580 2024-08-16T19:02:38.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.720+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f011819b650 con 0x7f0118196580 2024-08-16T19:02:38.721 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.721+0000 7f010f7fe640 1 -- 172.21.15.161:0/2152005000 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f01080172c0 con 0x7f0118196580 2024-08-16T19:02:38.721 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:02:38.722 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","modified":"2024-08-16T19:02:30.358525Z","created":"2024-08-16T19:00:59.803011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-16T19:02:38.722 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-08-16T19:02:38.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 msgr2=0x7f010403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:38.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 0x7f010403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f01140096c0 tx=0x7f0114005a40 comp rx=0 tx=0).stop 2024-08-16T19:02:38.723 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 msgr2=0x7f011819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 0x7f011819a8c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f010800b3d0 tx=0x7f010800b890 comp rx=0 tx=0).stop 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 shutdown_connections 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0104038e00 0x7f010403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0118196580 0x7f011819a8c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 --2- 172.21.15.161:0/2152005000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01181004f0 0x7f0118196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 >> 172.21.15.161:0/2152005000 conn(0x7f01180fbf50 msgr2=0x7f0118107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 shutdown_connections 2024-08-16T19:02:38.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:38.723+0000 7f012068b640 1 -- 172.21.15.161:0/2152005000 wait complete. 2024-08-16T19:02:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:39 smithi155 systemd[1]: Started Ceph mon.b for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:02:40.301 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-16T19:02:40.302 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-16T19:02:40.559 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:02:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 -- 172.21.15.161:0/3426551214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 msgr2=0x7f25000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:44.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 --2- 172.21.15.161:0/3426551214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f25000059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2518101720 tx=0x7f250802c150 comp rx=0 tx=0).stop 2024-08-16T19:02:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 -- 172.21.15.161:0/3426551214 shutdown_connections 2024-08-16T19:02:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 --2- 172.21.15.161:0/3426551214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f25000059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 -- 172.21.15.161:0/3426551214 >> 172.21.15.161:0/3426551214 conn(0x7f25180fbf50 msgr2=0x7f25180fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 -- 172.21.15.161:0/3426551214 shutdown_connections 2024-08-16T19:02:44.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.180+0000 7f251e407640 1 -- 172.21.15.161:0/3426551214 wait complete. 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.181+0000 7f251e407640 1 Processor -- start 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.181+0000 7f251e407640 1 -- start start 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f2518197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f25181083a0 con 0x7f2500003530 2024-08-16T19:02:44.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2518108220 con 0x7f25181975a0 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251e407640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2518108520 con 0x7f251819be70 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251c9a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251c9a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50274/0 (socket says 172.21.15.161:50274) 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f2517fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f2518197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f251c9a4640 1 -- 172.21.15.161:0/3877144904 learned_addr learned my addr 172.21.15.161:0/3877144904 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:44.184 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.182+0000 7f25177fe640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25177fe640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 msgr2=0x7f251819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-16T19:02:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25177fe640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 msgr2=0x7f251819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-16T19:02:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25177fe640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-16T19:02:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251c9a4640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 msgr2=0x7f251819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-08-16T19:02:44.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251c9a4640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 msgr2=0x7f251819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25177fe640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251c9a4640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251c9a4640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 msgr2=0x7f251819c320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 msgr2=0x7f251819b930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 -- 172.21.15.161:0/3877144904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25181a3be0 con 0x7f2500003530 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f2517fff640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f2518197060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2508036000 tx=0x7f2508004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508032040 con 0x7f2500003530 2024-08-16T19:02:44.186 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2508043070 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508036670 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25181a3db0 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.183+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181a4260 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.184+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2508036810 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.184+0000 7f25157fa640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 0x7f250403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.185+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f250806dea0 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.185+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25181094d0 con 0x7f2500003530 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.185+0000 7f25177fe640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 0x7f250403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:44.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.185+0000 7f25177fe640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 0x7f250403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f250c006fd0 tx=0x7f250c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:44.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.190+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2508045b80 con 0x7f2500003530 2024-08-16T19:02:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:02:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: mon.a calling monitor election 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: mon.c calling monitor election 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: mon.b calling monitor election 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: monmap epoch 3 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: last_changed 2024-08-16T19:02:39.048543+0000 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: min_mon_release 19 (squid) 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: election_strategy: 1 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: fsmap 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: mgrmap e13: x(active, since 46s) 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: overall HEALTH_OK 2024-08-16T19:02:44.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.437 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.436+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f251802d5e0 con 0x7f2500003530 2024-08-16T19:02:44.438 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.437+0000 7f25157fa640 1 -- 172.21.15.161:0/3877144904 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f250803d380 con 0x7f2500003530 2024-08-16T19:02:44.438 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:02:44.438 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","modified":"2024-08-16T19:02:39.048543Z","created":"2024-08-16T19:00:59.803011Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-16T19:02:44.438 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-08-16T19:02:44.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.439+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 msgr2=0x7f250403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:44.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.439+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 0x7f250403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f250c006fd0 tx=0x7f250c008040 comp rx=0 tx=0).stop 2024-08-16T19:02:44.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.439+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 msgr2=0x7f2518197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:44.440 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.439+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f2518197060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2508036000 tx=0x7f2508004d30 comp rx=0 tx=0).stop 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 shutdown_connections 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f251819be70 0x7f251819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25040394f0 0x7f250403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25181975a0 0x7f251819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 --2- 172.21.15.161:0/3877144904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2500003530 0x7f2518197060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 >> 172.21.15.161:0/3877144904 conn(0x7f25180fbf50 msgr2=0x7f251806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 shutdown_connections 2024-08-16T19:02:44.441 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:44.440+0000 7f251e407640 1 -- 172.21.15.161:0/3877144904 wait complete. 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: mon.a calling monitor election 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: mon.c calling monitor election 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: mon.b calling monitor election 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: monmap epoch 3 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: last_changed 2024-08-16T19:02:39.048543+0000 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: min_mon_release 19 (squid) 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: election_strategy: 1 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-16T19:02:44.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: fsmap 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: mgrmap e13: x(active, since 46s) 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: overall HEALTH_OK 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.499 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: mon.a calling monitor election 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: mon.c calling monitor election 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:02:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: mon.b calling monitor election 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: monmap epoch 3 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: last_changed 2024-08-16T19:02:39.048543+0000 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: created 2024-08-16T19:00:59.803011+0000 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: min_mon_release 19 (squid) 2024-08-16T19:02:44.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: election_strategy: 1 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: 0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: 2: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: fsmap 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: osdmap e4: 0 total, 0 up, 0 in 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: mgrmap e13: x(active, since 46s) 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: overall HEALTH_OK 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:44.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:45.039 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-16T19:02:45.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-16T19:02:45.310 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:45.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:45 smithi161 ceph-mon[37576]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:45.371 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:45 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/3877144904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.372 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:45.432 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:45 smithi062 ceph-mon[32993]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:45.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:45 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/3877144904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:45 smithi155 ceph-mon[37413]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:45 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/3877144904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-16T19:02:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 -- 172.21.15.62:0/1854030754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 msgr2=0x7fa5f8105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/1854030754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8105cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009880 tx=0x7fa5e802a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 -- 172.21.15.62:0/1854030754 shutdown_connections 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/1854030754 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8105cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 -- 172.21.15.62:0/1854030754 >> 172.21.15.62:0/1854030754 conn(0x7fa5f80fb5d0 msgr2=0x7fa5f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 -- 172.21.15.62:0/1854030754 shutdown_connections 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.741+0000 7fa5fcba1640 1 -- 172.21.15.62:0/1854030754 wait complete. 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 Processor -- start 2024-08-16T19:02:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 -- start start 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 0x7fa5f819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f819adc0 0x7fa5f819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8109f70 con 0x7fa5f8105910 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8109df0 con 0x7fa5f81964f0 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5fcba1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f810a0f0 con 0x7fa5f819adc0 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5f659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.742+0000 7fa5f659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34508/0 (socket says 172.21.15.62:34508) 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 -- 172.21.15.62:0/4228082679 learned_addr learned my addr 172.21.15.62:0/4228082679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5effff640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 0x7fa5f819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f6d9d640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f819adc0 0x7fa5f819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 -- 172.21.15.62:0/4228082679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f819adc0 msgr2=0x7fa5f819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f819adc0 0x7fa5f819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 -- 172.21.15.62:0/4228082679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 msgr2=0x7fa5f819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 0x7fa5f819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 -- 172.21.15.62:0/4228082679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f81a2b30 con 0x7fa5f8105910 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5effff640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 0x7fa5f819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5f659c640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8031550 tx=0x7fa5e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8046070 con 0x7fa5f8105910 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f80fb9b0 con 0x7fa5f8105910 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5e8003ad0 con 0x7fa5f8105910 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8035050 con 0x7fa5f8105910 2024-08-16T19:02:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.743+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f80fbef0 con 0x7fa5f8105910 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.744+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa5e8049070 con 0x7fa5f8105910 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.744+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f8101050 con 0x7fa5f8105910 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.744+0000 7fa5ef7fe640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 0x7fa5e403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.744+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa5e806e1c0 con 0x7fa5f8105910 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.745+0000 7fa5effff640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 0x7fa5e403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.745+0000 7fa5effff640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 0x7fa5e403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009890 tx=0x7fa5e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.749+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa5e8030560 con 0x7fa5f8105910 2024-08-16T19:02:45.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.930+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa5f81045a0 con 0x7fa5f8105910 2024-08-16T19:02:45.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.931+0000 7fa5ef7fe640 1 -- 172.21.15.62:0/4228082679 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fa5e8003c80 con 0x7fa5f8105910 2024-08-16T19:02:45.932 INFO:teuthology.orchestra.run.smithi062.stdout:# minimal ceph.conf for bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:45.932 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-08-16T19:02:45.932 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:02:45.933 INFO:teuthology.orchestra.run.smithi062.stdout: mon_host = [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 msgr2=0x7fa5e403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 0x7fa5e403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009890 tx=0x7fa5e0008040 comp rx=0 tx=0).stop 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 msgr2=0x7fa5f8195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8031550 tx=0x7fa5e8004270 comp rx=0 tx=0).stop 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 shutdown_connections 2024-08-16T19:02:45.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5f819adc0 0x7fa5f819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa5e40394f0 0x7fa5e403b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5f81964f0 0x7fa5f819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 --2- 172.21.15.62:0/4228082679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5f8105910 0x7fa5f8195fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 >> 172.21.15.62:0/4228082679 conn(0x7fa5f80fb5d0 msgr2=0x7fa5f80fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 shutdown_connections 2024-08-16T19:02:45.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:45.933+0000 7fa5fcba1640 1 -- 172.21.15.62:0/4228082679 wait complete. 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:46.452 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4228082679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4228082679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:46.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4228082679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:46.680 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-16T19:02:46.680 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:46.681 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-08-16T19:02:46.709 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:46.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:46.777 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:02:46.777 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-08-16T19:02:46.804 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:02:46.804 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:46.867 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:02:46.867 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-08-16T19:02:46.893 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:02:46.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:02:46.958 INFO:tasks.cephadm:Adding mgr.x on smithi062 2024-08-16T19:02:46.958 INFO:tasks.cephadm:Adding mgr.y on smithi155 2024-08-16T19:02:46.958 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-08-16T19:02:46.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi062=x;smithi155=y;smithi161=z' 2024-08-16T19:02:47.240 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Reconfiguring mon.a (unknown last config time)... 2024-08-16T19:02:47.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:47.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:47.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:47.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:47 smithi155 ceph-mon[37413]: Reconfiguring daemon mon.a on smithi062 2024-08-16T19:02:47.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:47.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:02:47.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:02:47.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:02:47.601 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Reconfiguring mon.a (unknown last config time)... 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:47.603 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:47 smithi161 ceph-mon[37576]: Reconfiguring daemon mon.a on smithi062 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 -- 172.21.15.161:0/1289926840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 msgr2=0x7f612c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 --2- 172.21.15.161:0/1289926840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c100720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f611c009880 tx=0x7f611c02a5d0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 -- 172.21.15.161:0/1289926840 shutdown_connections 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 --2- 172.21.15.161:0/1289926840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c100720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 -- 172.21.15.161:0/1289926840 >> 172.21.15.161:0/1289926840 conn(0x7f612c0fbad0 msgr2=0x7f612c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 -- 172.21.15.161:0/1289926840 shutdown_connections 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.681+0000 7f613286a640 1 -- 172.21.15.161:0/1289926840 wait complete. 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 Processor -- start 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 -- start start 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c06acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:47.735 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612c199a60 0x7f612c19be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f612c106410 con 0x7f612c100320 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f612c106290 con 0x7f612c199a60 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f613286a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f612c106590 con 0x7f612c19c390 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f6130e07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f6130e07640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50276/0 (socket says 172.21.15.161:50276) 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f6130e07640 1 -- 172.21.15.161:0/2329236223 learned_addr learned my addr 172.21.15.161:0/2329236223 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f612b7fe640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612c199a60 0x7f612c19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.682+0000 7f612bfff640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c06acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 -- 172.21.15.161:0/2329236223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612c199a60 msgr2=0x7f612c19be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612c199a60 0x7f612c19be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 -- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 msgr2=0x7f612c06acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:47.736 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c06acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 -- 172.21.15.161:0/2329236223 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c19f160 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f612bfff640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c06acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f6130e07640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f612400ba50 tx=0x7f612400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124019070 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612400cbe0 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240076a0 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c19f3f0 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.683+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c19f960 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.684+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f612401c070 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.684+0000 7f61297fa640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 0x7f611403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.684+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c18f1d0 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.684+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f612404bab0 con 0x7f612c19c390 2024-08-16T19:02:47.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.684+0000 7f612bfff640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 0x7f611403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:47.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.685+0000 7f612bfff640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 0x7f611403b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f611c031440 tx=0x7f611c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:47.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.689+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f612400e0a0 con 0x7f612c19c390 2024-08-16T19:02:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:02:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:02:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:02:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Reconfiguring mon.a (unknown last config time)... 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:47.749 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:47 smithi062 ceph-mon[32993]: Reconfiguring daemon mon.a on smithi062 2024-08-16T19:02:47.868 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.868+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi155=y;smithi161=z", "target": ["mon-mgr", ""]}) -- 0x7f612c0623f0 con 0x7f61140394f0 2024-08-16T19:02:47.873 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.873+0000 7f61297fa640 1 -- 172.21.15.161:0/2329236223 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f612c0623f0 con 0x7f61140394f0 2024-08-16T19:02:47.873 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-08-16T19:02:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 msgr2=0x7f611403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 0x7f611403b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f611c031440 tx=0x7f611c0058d0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 msgr2=0x7f612c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f612400ba50 tx=0x7f612400bf10 comp rx=0 tx=0).stop 2024-08-16T19:02:47.876 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 shutdown_connections 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f612c19c390 0x7f612c19e850 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f61140394f0 0x7f611403b9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f612c199a60 0x7f612c19be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 --2- 172.21.15.161:0/2329236223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f612c100320 0x7f612c06acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 >> 172.21.15.161:0/2329236223 conn(0x7f612c0fbad0 msgr2=0x7f612c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.875+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 shutdown_connections 2024-08-16T19:02:47.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:02:47.876+0000 7f613286a640 1 -- 172.21.15.161:0/2329236223 wait complete. 2024-08-16T19:02:48.425 DEBUG:teuthology.orchestra.run.smithi155:mgr.y> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.y.service 2024-08-16T19:02:48.427 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.z.service 2024-08-16T19:02:48.430 INFO:tasks.cephadm:Deploying OSDs... 2024-08-16T19:02:48.430 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:02:48.430 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-08-16T19:02:48.451 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-16T19:02:48.451 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-08-16T19:02:48.510 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-16T19:02:48.510 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:48.510 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 673 Links: 1 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-08-16 19:02:08.155182306 +0000 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-08-16 18:57:21.284488857 +0000 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-08-16 18:57:21.284488857 +0000 2024-08-16T19:02:48.511 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-08-16 18:57:21.284488857 +0000 2024-08-16T19:02:48.511 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-16T19:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-08-16T19:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-08-16T19:02:48.581 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00021966 s, 2.3 MB/s 2024-08-16T19:02:48.583 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-16T19:02:48.642 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-08-16T19:02:48.699 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-16T19:02:48.699 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-08-16 19:02:08.191181727 +0000 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-08-16 18:57:21.624483410 +0000 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-08-16 18:57:21.624483410 +0000 2024-08-16T19:02:48.700 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-08-16 18:57:21.624483410 +0000 2024-08-16T19:02:48.700 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-16T19:02:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-08-16T19:02:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-08-16T19:02:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000190902 s, 2.7 MB/s 2024-08-16T19:02:48.766 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-16T19:02:48.824 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: Reconfiguring mon.b (monmap changed)... 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: Reconfiguring daemon mon.b on smithi155 2024-08-16T19:02:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi155=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:48.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: Saving service mgr spec with placement smithi062=x;smithi155=y;smithi161=z;count:3 2024-08-16T19:02:48.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.881 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 710 Links: 1 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-08-16 19:02:08.227181149 +0000 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-08-16 18:57:21.972477836 +0000 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-08-16 18:57:21.972477836 +0000 2024-08-16T19:02:48.882 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-08-16 18:57:21.972477836 +0000 2024-08-16T19:02:48.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: Reconfiguring mon.b (monmap changed)... 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: Reconfiguring daemon mon.b on smithi155 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi155=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: Saving service mgr spec with placement smithi062=x;smithi155=y;smithi161=z;count:3 2024-08-16T19:02:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:48.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: Reconfiguring mon.b (monmap changed)... 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: Reconfiguring daemon mon.b on smithi155 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi062=x;smithi155=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: Saving service mgr spec with placement smithi062=x;smithi155=y;smithi161=z;count:3 2024-08-16T19:02:48.945 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-08-16T19:02:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-08-16T19:02:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000222279 s, 2.3 MB/s 2024-08-16T19:02:48.948 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-16T19:02:49.005 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 728 Links: 1 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-08-16 19:02:08.264180554 +0000 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-08-16 18:57:22.321472245 +0000 2024-08-16T19:02:49.062 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-08-16 18:57:22.321472245 +0000 2024-08-16T19:02:49.063 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-08-16 18:57:22.321472245 +0000 2024-08-16T19:02:49.063 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-16T19:02:49.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-08-16T19:02:49.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-08-16T19:02:49.127 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000196291 s, 2.6 MB/s 2024-08-16T19:02:49.128 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-16T19:02:49.185 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-08-16T19:02:49.185 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-08-16T19:02:49.200 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-16T19:02:49.200 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-08-16T19:02:49.255 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-16T19:02:49.255 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.255 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-16 19:02:38.536849303 +0000 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-16 18:57:25.886241393 +0000 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-16 18:57:25.886241393 +0000 2024-08-16T19:02:49.256 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-16 18:57:25.886241393 +0000 2024-08-16T19:02:49.256 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-16T19:02:49.318 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-16T19:02:49.318 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-16T19:02:49.319 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000266432 s, 1.9 MB/s 2024-08-16T19:02:49.319 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-16T19:02:49.375 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-08-16T19:02:49.430 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-16T19:02:49.430 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-16 19:02:38.536849303 +0000 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-16 18:57:26.219234262 +0000 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-16 18:57:26.219234262 +0000 2024-08-16T19:02:49.431 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-16 18:57:26.219234262 +0000 2024-08-16T19:02:49.431 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-16T19:02:49.492 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-16T19:02:49.492 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-16T19:02:49.493 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000222754 s, 2.3 MB/s 2024-08-16T19:02:49.494 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-16T19:02:49.549 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-16 19:02:38.536849303 +0000 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-16 18:57:26.550227174 +0000 2024-08-16T19:02:49.604 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-16 18:57:26.550227174 +0000 2024-08-16T19:02:49.605 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-16 18:57:26.550227174 +0000 2024-08-16T19:02:49.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-16T19:02:49.666 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-16T19:02:49.666 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-16T19:02:49.666 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000195511 s, 2.6 MB/s 2024-08-16T19:02:49.667 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-16T19:02:49.723 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.777 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-08-16 19:02:38.537849282 +0000 2024-08-16T19:02:49.778 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-08-16 18:57:26.908219508 +0000 2024-08-16T19:02:49.778 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-08-16 18:57:26.908219508 +0000 2024-08-16T19:02:49.778 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-08-16 18:57:26.908219508 +0000 2024-08-16T19:02:49.778 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-16T19:02:49.839 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-08-16T19:02:49.840 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-08-16T19:02:49.840 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000175873 s, 2.9 MB/s 2024-08-16T19:02:49.841 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-16T19:02:49.896 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-16T19:02:49.896 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-08-16T19:02:49.912 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-16T19:02:49.912 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:49.968 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-16 19:02:29.477479869 +0000 2024-08-16T19:02:49.969 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-16 18:57:38.750635663 +0000 2024-08-16T19:02:49.969 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-16 18:57:38.750635663 +0000 2024-08-16T19:02:49.969 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-16 18:57:38.750635663 +0000 2024-08-16T19:02:49.969 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-16T19:02:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-16T19:02:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-16T19:02:50.032 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00027202 s, 1.9 MB/s 2024-08-16T19:02:50.033 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-16T19:02:50.089 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-08-16T19:02:50.144 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-16T19:02:50.144 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-16 19:02:29.477479869 +0000 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-16 18:57:39.083628251 +0000 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-16 18:57:39.083628251 +0000 2024-08-16T19:02:50.145 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-16 18:57:39.083628251 +0000 2024-08-16T19:02:50.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-16T19:02:50.208 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-16T19:02:50.208 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-16T19:02:50.208 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000177141 s, 2.9 MB/s 2024-08-16T19:02:50.209 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-16T19:02:50.265 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-08-16T19:02:50.320 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-16T19:02:50.320 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-16 19:02:29.477479869 +0000 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-16 18:57:39.414620883 +0000 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-16 18:57:39.414620883 +0000 2024-08-16T19:02:50.321 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-16 18:57:39.414620883 +0000 2024-08-16T19:02:50.321 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-16T19:02:50.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: Reconfiguring mon.c (monmap changed)... 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:50 smithi155 ceph-mon[37413]: Reconfiguring daemon mon.c on smithi161 2024-08-16T19:02:50.380 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.380 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.380 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: Reconfiguring mon.c (monmap changed)... 2024-08-16T19:02:50.380 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:50.380 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:50.381 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:50 smithi161 ceph-mon[37576]: Reconfiguring daemon mon.c on smithi161 2024-08-16T19:02:50.382 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-16T19:02:50.382 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-16T19:02:50.383 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000193915 s, 2.6 MB/s 2024-08-16T19:02:50.384 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-16T19:02:50.441 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-16 19:02:29.478479848 +0000 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-16 18:57:39.755613293 +0000 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-16 18:57:39.755613293 +0000 2024-08-16T19:02:50.496 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-08-16 18:57:39.755613293 +0000 2024-08-16T19:02:50.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: Reconfiguring mon.c (monmap changed)... 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-16T19:02:50.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:50.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:50 smithi062 ceph-mon[32993]: Reconfiguring daemon mon.c on smithi161 2024-08-16T19:02:50.561 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-16T19:02:50.561 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-16T19:02:50.561 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000211306 s, 2.4 MB/s 2024-08-16T19:02:50.562 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-16T19:02:50.617 INFO:tasks.cephadm:Deploying osd.0 on smithi062 with /dev/vg_nvme/lv_4... 2024-08-16T19:02:50.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-16T19:02:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.258 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:51.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:51.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:51.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.259 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:51 smithi062 ceph-mon[32993]: Deploying daemon mgr.z on smithi161 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:51.551 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:51.551 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:51.551 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.551 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:51 smithi161 ceph-mon[37576]: Deploying daemon mgr.z on smithi161 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:51.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:51 smithi155 ceph-mon[37413]: Deploying daemon mgr.z on smithi161 2024-08-16T19:02:52.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:51 smithi161 systemd[1]: Starting Ceph mgr.z for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:02:52.518 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 podman[39370]: 2024-08-16 19:02:52.202328114 +0000 UTC m=+0.105619643 container create 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:02:52.518 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 podman[39370]: 2024-08-16 19:02:52.114134965 +0000 UTC m=+0.017426471 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:02:52.518 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 podman[39370]: 2024-08-16 19:02:52.387900547 +0000 UTC m=+0.291192045 container init 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-08-16T19:02:52.518 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 podman[39370]: 2024-08-16 19:02:52.3905298 +0000 UTC m=+0.293821290 container start 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:02:52.518 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 ceph-mgr[39390]: -- 172.21.15.161:0/246329832 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560798d50e0 con 0x5560798b2800 2024-08-16T19:02:52.519 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 bash[39370]: 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f 2024-08-16T19:02:52.519 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 systemd[1]: Started Ceph mgr.z for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:02:52.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:52.516+0000 7fbe211cb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:02:52.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:52.662+0000 7fbe211cb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:02:52.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:52.729+0000 7fbe211cb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:02:53.179 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:02:53.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-08-16T19:02:53.388 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:52.970+0000 7fbe211cb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:02:53.388 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:53.028+0000 7fbe211cb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:02:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:02:53.760 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:53.761 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:53 smithi062 ceph-mon[32993]: Deploying daemon mgr.y on smithi155 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:53 smithi155 ceph-mon[37413]: Deploying daemon mgr.y on smithi155 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-mon[37576]: Deploying daemon mgr.y on smithi155 2024-08-16T19:02:53.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 -- 172.21.15.62:0/569437670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 msgr2=0x7f63a0102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:53.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 --2- 172.21.15.62:0/569437670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0102fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6394009900 tx=0x7f639402a410 comp rx=0 tx=0).stop 2024-08-16T19:02:53.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 -- 172.21.15.62:0/569437670 shutdown_connections 2024-08-16T19:02:53.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 --2- 172.21.15.62:0/569437670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 0x7f63a0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:53.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 --2- 172.21.15.62:0/569437670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0102fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:53.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 --2- 172.21.15.62:0/569437670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:53.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 -- 172.21.15.62:0/569437670 >> 172.21.15.62:0/569437670 conn(0x7f63a00fd0c0 msgr2=0x7f63a00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:02:53.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 -- 172.21.15.62:0/569437670 shutdown_connections 2024-08-16T19:02:53.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.931+0000 7f63a59f8640 1 -- 172.21.15.62:0/569437670 wait complete. 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.932+0000 7f63a59f8640 1 Processor -- start 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.932+0000 7f63a59f8640 1 -- start start 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 0x7f63a019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f63a010d5b0 con 0x7f63a0102b30 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f63a010d430 con 0x7f63a01034f0 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f63a59f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f63a010d730 con 0x7f63a0101930 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:52410/0 (socket says 172.21.15.62:52410) 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639effd640 1 -- 172.21.15.62:0/1190319667 learned_addr learned my addr 172.21.15.62:0/1190319667 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40668/0 (socket says 172.21.15.62:40668) 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639f7fe640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 0x7f63a019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 -- 172.21.15.62:0/1190319667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 msgr2=0x7f63a0197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 -- 172.21.15.62:0/1190319667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 msgr2=0x7f63a019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 0x7f63a019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 -- 172.21.15.62:0/1190319667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a019f830 con 0x7f63a0102b30 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639effd640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.933+0000 7f639e7fc640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6394031440 tx=0x7f6394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:53.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.934+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639403f070 con 0x7f63a0102b30 2024-08-16T19:02:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.934+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a019fac0 con 0x7f63a0102b30 2024-08-16T19:02:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.934+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6394003cd0 con 0x7f63a0102b30 2024-08-16T19:02:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.934+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394035070 con 0x7f63a0102b30 2024-08-16T19:02:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.934+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a019ffc0 con 0x7f63a0102b30 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.935+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6394043610 con 0x7f63a0102b30 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.935+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a0108cd0 con 0x7f63a0102b30 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.935+0000 7f63a49f6640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 0x7f638803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.935+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f639406cb40 con 0x7f63a0102b30 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.935+0000 7f639effd640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 0x7f638803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:02:53.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.936+0000 7f639effd640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 0x7f638803b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63a01990f0 tx=0x7f6390009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:02:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:53.940+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f639403be20 con 0x7f63a0102b30 2024-08-16T19:02:54.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:54.125+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f63a0062360 con 0x7f6388039290 2024-08-16T19:02:54.356 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:53.973+0000 7fbe211cb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:02:54.356 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.086+0000 7fbe211cb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:02:54.356 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.142+0000 7fbe211cb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:02:54.356 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.354+0000 7fbe211cb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:02:54.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:02:54.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:02:54.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:54.629 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.416+0000 7fbe211cb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:02:54.630 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.483+0000 7fbe211cb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:02:54.630 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:54.627+0000 7fbe211cb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:02:54.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:02:54.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:02:54.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:54.749 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 systemd[1]: Starting Ceph mgr.y for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:02:54.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:02:54.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:02:54.824 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:55.009 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 podman[38770]: 2024-08-16 19:02:54.748514285 +0000 UTC m=+0.099742106 container create c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-08-16T19:02:55.010 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 podman[38770]: 2024-08-16 19:02:54.666500508 +0000 UTC m=+0.017728298 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:02:55.010 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 podman[38770]: 2024-08-16 19:02:54.950720539 +0000 UTC m=+0.301948335 container init c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-16T19:02:55.010 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 podman[38770]: 2024-08-16 19:02:54.953398035 +0000 UTC m=+0.304625821 container start c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.build-date=20240731) 2024-08-16T19:02:55.010 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:54 smithi155 ceph-mgr[38789]: -- 172.21.15.155:0/3528076414 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582852c70e0 con 0x5582852a4800 2024-08-16T19:02:55.010 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 bash[38770]: c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c 2024-08-16T19:02:55.312 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 systemd[1]: Started Ceph mgr.y for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:02:55.312 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:55.088+0000 7f95bdc32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:02:55.313 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:55.241+0000 7f95bdc32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:02:55.313 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:55.311+0000 7f95bdc32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:02:55.367 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:55.040+0000 7fbe211cb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:02:55.367 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:55.365+0000 7fbe211cb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:55.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:55.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:55.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:55.436+0000 7fbe211cb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:02:55.623 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:55.620+0000 7fbe211cb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:02:55.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:55.816 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:02:55.901 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:55.588+0000 7f95bdc32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:02:55.902 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:55.649+0000 7f95bdc32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:02:56.195 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:55.944+0000 7fbe211cb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:02:56.196 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.007+0000 7fbe211cb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:02:56.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.194+0000 7fbe211cb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:02:56.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.373+0000 7fbe211cb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:02:56.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.440+0000 7fbe211cb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:02:56.527 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.524+0000 7fbe211cb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:02:56.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:56.584+0000 7fbe211cb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:02:57.006 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:56.626+0000 7f95bdc32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:02:57.006 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:56.739+0000 7f95bdc32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:02:57.006 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:56.796+0000 7f95bdc32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:02:57.006 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.005+0000 7f95bdc32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:02:57.278 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.066+0000 7f95bdc32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.131+0000 7f95bdc32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.277+0000 7f95bdc32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.279 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]': finished 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: osdmap e5: 1 total, 0 up, 1 in 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: Reconfiguring mgr.x (unknown last config time)... 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: Reconfiguring daemon mgr.x on smithi062 2024-08-16T19:02:57.280 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/422746912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:57.117+0000 7fbe211cb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:02:57.276+0000 7fbe211cb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]': finished 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: osdmap e5: 1 total, 0 up, 1 in 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: Reconfiguring mgr.x (unknown last config time)... 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: Reconfiguring daemon mgr.x on smithi062 2024-08-16T19:02:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/422746912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:02:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]: dispatch 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4211392186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0cec841-7b80-4f4a-909c-74544c63b869"}]': finished 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: osdmap e5: 1 total, 0 up, 1 in 2024-08-16T19:02:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: Reconfiguring mgr.x (unknown last config time)... 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: Reconfiguring daemon mgr.x on smithi062 2024-08-16T19:02:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/422746912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:02:58.000 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.690+0000 7f95bdc32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:02:58.000 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:57.999+0000 7f95bdc32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:02:58.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:02:58.244+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f639403a740 con 0x7f63a0102b30 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.065+0000 7f95bdc32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.213+0000 7f95bdc32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: Standby manager daemon z started 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:02:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: Standby manager daemon z started 2024-08-16T19:02:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:02:58.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:02:58.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:02:58.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:02:58.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: Standby manager daemon z started 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/685053908' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:02:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:02:58.829 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.523+0000 7f95bdc32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:02:58.829 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.584+0000 7f95bdc32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:02:58.829 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.765+0000 7f95bdc32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:02:59.242 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.911+0000 7f95bdc32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:02:59.243 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:58.974+0000 7f95bdc32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:02:59.243 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:59.055+0000 7f95bdc32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:02:59.243 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:59.114+0000 7f95bdc32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:02:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:59 smithi062 ceph-mon[32993]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:59 smithi062 ceph-mon[32993]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-16T19:02:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:02:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:02:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-mon[37413]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-mon[37413]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-16T19:02:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:02:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:59 smithi161 ceph-mon[37576]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:02:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:59 smithi161 ceph-mon[37576]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-16T19:02:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:02:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:03:00.079 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:59.646+0000 7f95bdc32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:03:00.079 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:02:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:02:59.803+0000 7f95bdc32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:03:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:00.253+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f639403b4f0 con 0x7f63a0102b30 2024-08-16T19:03:00.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:00 smithi062 ceph-mon[32993]: Standby manager daemon y started 2024-08-16T19:03:00.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:00 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:03:00.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:00 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:03:00.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:00 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:03:00.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:00 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:03:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:00 smithi155 ceph-mon[37413]: Standby manager daemon y started 2024-08-16T19:03:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:00 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:03:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:00 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:03:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:00 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:03:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:00 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:03:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:00 smithi161 ceph-mon[37576]: Standby manager daemon y started 2024-08-16T19:03:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:00 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:03:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:00 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:03:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:00 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:03:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:00 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1862965506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:03:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:01 smithi155 ceph-mon[37413]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:01 smithi155 ceph-mon[37413]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-16T19:03:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:03:01.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:01 smithi161 ceph-mon[37576]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:01.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:01 smithi161 ceph-mon[37576]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-16T19:03:01.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:03:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:01 smithi062 ceph-mon[32993]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:01 smithi062 ceph-mon[32993]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-16T19:03:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:03.179 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:04.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-16T19:03:04.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:04.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-16T19:03:04.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:04.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-16T19:03:04.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:05.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:04 smithi062 ceph-mon[32993]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:05.036 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:04 smithi062 ceph-mon[32993]: Deploying daemon osd.0 on smithi062 2024-08-16T19:03:05.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:04 smithi161 ceph-mon[37576]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:05.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:04 smithi161 ceph-mon[37576]: Deploying daemon osd.0 on smithi062 2024-08-16T19:03:05.229 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:04 smithi155 ceph-mon[37413]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:05.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:04 smithi155 ceph-mon[37413]: Deploying daemon osd.0 on smithi062 2024-08-16T19:03:07.130 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:06 smithi062 ceph-mon[32993]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:07.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:06 smithi161 ceph-mon[37576]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:07.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:06 smithi155 ceph-mon[37413]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:09.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:08 smithi161 ceph-mon[37576]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:09.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:09.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:09.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:09.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:08 smithi062 ceph-mon[32993]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:09.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:09.164 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:09.165 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:09.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:08 smithi155 ceph-mon[37413]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:09.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:09.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:09.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:11.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:10 smithi161 ceph-mon[37576]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:10 smithi062 ceph-mon[32993]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:10 smithi155 ceph-mon[37413]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:11.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.688+0000 7f63a49f6640 1 -- 172.21.15.62:0/1190319667 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f63a0062360 con 0x7f6388039290 2024-08-16T19:03:11.689 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 0 on host 'smithi062' 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 msgr2=0x7f638803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 0x7f638803b750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f63a01990f0 tx=0x7f6390009500 comp rx=0 tx=0).stop 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 msgr2=0x7f63a0198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6394031440 tx=0x7f6394004270 comp rx=0 tx=0).stop 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 shutdown_connections 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f6388039290 0x7f638803b750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63a01034f0 0x7f63a019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a0102b30 0x7f63a0198150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:11.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 --2- 172.21.15.62:0/1190319667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0101930 0x7f63a0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 >> 172.21.15.62:0/1190319667 conn(0x7f63a00fd0c0 msgr2=0x7f63a0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 shutdown_connections 2024-08-16T19:03:11.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:11.690+0000 7f63a59f8640 1 -- 172.21.15.62:0/1190319667 wait complete. 2024-08-16T19:03:12.222 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.223 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.253 DEBUG:teuthology.orchestra.run.smithi062:osd.0> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.0.service 2024-08-16T19:03:12.255 INFO:tasks.cephadm:Deploying osd.1 on smithi155 with /dev/vg_nvme/lv_4... 2024-08-16T19:03:12.255 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:12.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:12.496 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:03:12.330+0000 7fbe86547640 -1 osd.0 0 log_to_monitors true 2024-08-16T19:03:12.515 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:03:13.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:12 smithi062 ceph-mon[32993]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:13.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:12 smithi062 ceph-mon[32993]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-16T19:03:13.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:12 smithi155 ceph-mon[37413]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:13.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:12 smithi155 ceph-mon[37413]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-16T19:03:13.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:12 smithi161 ceph-mon[37576]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:13.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:12 smithi161 ceph-mon[37576]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-16T19:03:14.246 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:03:13.989+0000 7fbe824ca640 -1 osd.0 0 waiting for initial osdmap 2024-08-16T19:03:14.246 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:03:13.992+0000 7fbe7e2f2640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:03:14.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-mon[32993]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-16T19:03:14.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-mon[32993]: osdmap e6: 1 total, 0 up, 1 in 2024-08-16T19:03:14.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-mon[32993]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:03:14.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:13 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:13 smithi155 ceph-mon[37413]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-16T19:03:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:13 smithi155 ceph-mon[37413]: osdmap e6: 1 total, 0 up, 1 in 2024-08-16T19:03:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:13 smithi155 ceph-mon[37413]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:03:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:13 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:13 smithi161 ceph-mon[37576]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-16T19:03:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:13 smithi161 ceph-mon[37576]: osdmap e6: 1 total, 0 up, 1 in 2024-08-16T19:03:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:13 smithi161 ceph-mon[37576]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:03:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:13 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:14.584 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T19:03:14.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-08-16T19:03:14.899 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:03:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:03:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: osdmap e7: 1 total, 0 up, 1 in 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: osdmap e7: 1 total, 0 up, 1 in 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:15.294 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:15.295 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/1605913634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc104d00 msgr2=0x7f0fcc1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/1605913634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc104d00 0x7f0fcc1070f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009900 tx=0x7f0fb402a410 comp rx=0 tx=0).stop 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/1605913634 shutdown_connections 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/1605913634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc107630 0x7f0fcc109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/1605913634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc104d00 0x7f0fcc1070f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/1605913634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc1023d0 0x7f0fcc1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/1605913634 >> 172.21.15.155:0/1605913634 conn(0x7f0fcc0fbfd0 msgr2=0x7f0fcc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/1605913634 shutdown_connections 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.373+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/1605913634 wait complete. 2024-08-16T19:03:15.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 Processor -- start 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 -- start start 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc1023d0 0x7f0fcc197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc104d00 0x7f0fcc198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc10d6f0 con 0x7f0fcc104d00 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc10d570 con 0x7f0fcc107630 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fd0ea9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc10d870 con 0x7f0fcc1023d0 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fcad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fcad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:48406/0 (socket says 172.21.15.155:48406) 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fc1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc104d00 0x7f0fcc198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fca59c640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc1023d0 0x7f0fcc197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:15.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fcad9d640 1 -- 172.21.15.155:0/2271908975 learned_addr learned my addr 172.21.15.155:0/2271908975 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.374+0000 7f0fcad9d640 1 -- 172.21.15.155:0/2271908975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc1023d0 msgr2=0x7f0fcc197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fcad9d640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc1023d0 0x7f0fcc197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fcad9d640 1 -- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc104d00 msgr2=0x7f0fcc198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fcad9d640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc104d00 0x7f0fcc198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fcad9d640 1 -- 172.21.15.155:0/2271908975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc19fa30 con 0x7f0fcc107630 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fcad9d640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b970 tx=0x7f0fc400bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4019070 con 0x7f0fcc107630 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fc4014440 con 0x7f0fcc107630 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc19fd20 con 0x7f0fcc107630 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40075d0 con 0x7f0fcc107630 2024-08-16T19:03:15.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.375+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc1a02a0 con 0x7f0fcc107630 2024-08-16T19:03:15.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.376+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc0fdac0 con 0x7f0fcc107630 2024-08-16T19:03:15.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.377+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0fc401c070 con 0x7f0fcc107630 2024-08-16T19:03:15.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.377+0000 7f0fc3fff640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 0x7f0fb80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:15.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.378+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f0fc40cca20 con 0x7f0fcc107630 2024-08-16T19:03:15.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.378+0000 7f0fca59c640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 0x7f0fb80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:15.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.378+0000 7f0fca59c640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 0x7f0fb80a5060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009990 tx=0x7f0fbc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:15.382 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.382+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0fc409ada0 con 0x7f0fcc107630 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: osdmap e7: 1 total, 0 up, 1 in 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:15.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:15.557+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0fcc062360 con 0x7f0fb80a2ba0 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi062 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 17088M 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] boot 2024-08-16T19:03:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: osdmap e8: 1 total, 1 up, 1 in 2024-08-16T19:03:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi062 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 17088M 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] boot 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: osdmap e8: 1 total, 1 up, 1 in 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:16.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:16.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:03:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi062 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 17088M 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: osd.0 [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] boot 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: osdmap e8: 1 total, 1 up, 1 in 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:16.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:17.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:16 smithi155 ceph-mon[37413]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:17.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:16 smithi155 ceph-mon[37413]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:17.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:16 smithi155 ceph-mon[37413]: osdmap e9: 1 total, 1 up, 1 in 2024-08-16T19:03:17.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:16 smithi161 ceph-mon[37576]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:17.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:16 smithi161 ceph-mon[37576]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:17.300 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:16 smithi161 ceph-mon[37576]: osdmap e9: 1 total, 1 up, 1 in 2024-08-16T19:03:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:16 smithi062 ceph-mon[32993]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:16 smithi062 ceph-mon[32993]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:16 smithi062 ceph-mon[32993]: osdmap e9: 1 total, 1 up, 1 in 2024-08-16T19:03:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:18 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/2031830703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:18 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:18 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]': finished 2024-08-16T19:03:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:18 smithi155 ceph-mon[37413]: osdmap e10: 2 total, 1 up, 2 in 2024-08-16T19:03:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:18 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/2031830703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:18 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:18 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]': finished 2024-08-16T19:03:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:18 smithi161 ceph-mon[37576]: osdmap e10: 2 total, 1 up, 2 in 2024-08-16T19:03:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:18.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:18 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/2031830703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:18 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]: dispatch 2024-08-16T19:03:18.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:18 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54e23d69-41c1-420c-aa46-2833c651059e"}]': finished 2024-08-16T19:03:18.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:18 smithi062 ceph-mon[32993]: osdmap e10: 2 total, 1 up, 2 in 2024-08-16T19:03:18.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:19.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:19 smithi155 ceph-mon[37413]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:19.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:19 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/1942350329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:19.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:19 smithi161 ceph-mon[37576]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:19.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:19 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/1942350329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:19.398 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:19 smithi062 ceph-mon[32993]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:19.398 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:19 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/1942350329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:21.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:21 smithi155 ceph-mon[37413]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:21.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:21 smithi161 ceph-mon[37576]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:21.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:21 smithi062 ceph-mon[32993]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:23.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:23 smithi155 ceph-mon[37413]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:23.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:23 smithi161 ceph-mon[37576]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:23.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:23 smithi062 ceph-mon[32993]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:25.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:25 smithi155 ceph-mon[37413]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:25.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:25 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-16T19:03:25.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:25 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:25.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:25 smithi155 ceph-mon[37413]: Deploying daemon osd.1 on smithi155 2024-08-16T19:03:25.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:25 smithi161 ceph-mon[37576]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:25.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:25 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-16T19:03:25.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:25 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:25.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:25 smithi161 ceph-mon[37576]: Deploying daemon osd.1 on smithi155 2024-08-16T19:03:25.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:25 smithi062 ceph-mon[32993]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:25.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-16T19:03:25.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:25.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:25 smithi062 ceph-mon[32993]: Deploying daemon osd.1 on smithi155 2024-08-16T19:03:27.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:27 smithi155 ceph-mon[37413]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:27 smithi161 ceph-mon[37576]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:27.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:27 smithi062 ceph-mon[32993]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:29.067 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:29 smithi155 ceph-mon[37413]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:29.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:29 smithi161 ceph-mon[37576]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:29 smithi062 ceph-mon[32993]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:30.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:30.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:30.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:30.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:30.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:30.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:31 smithi155 ceph-mon[37413]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:31.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:31 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:31 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:31 smithi161 ceph-mon[37576]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:31 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:31 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:31 smithi062 ceph-mon[32993]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:31 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:31 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:32.323 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:32.324 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:32.324 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:32.362 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.361+0000 7f0fc3fff640 1 -- 172.21.15.155:0/2271908975 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0fcc062360 con 0x7f0fb80a2ba0 2024-08-16T19:03:32.362 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 1 on host 'smithi155' 2024-08-16T19:03:32.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 msgr2=0x7f0fb80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:32.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 0x7f0fb80a5060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009990 tx=0x7f0fbc009110 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 msgr2=0x7f0fcc19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b970 tx=0x7f0fc400bd70 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 shutdown_connections 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0fb80a2ba0 0x7f0fb80a5060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fcc107630 0x7f0fcc19f3c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fcc104d00 0x7f0fcc198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 --2- 172.21.15.155:0/2271908975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fcc1023d0 0x7f0fcc197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:32.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.363+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 >> 172.21.15.155:0/2271908975 conn(0x7f0fcc0fbfd0 msgr2=0x7f0fcc102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.364+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 shutdown_connections 2024-08-16T19:03:32.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:03:32.364+0000 7f0fd0ea9640 1 -- 172.21.15.155:0/2271908975 wait complete. 2024-08-16T19:03:32.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:32.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:32.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:32.953 DEBUG:teuthology.orchestra.run.smithi155:osd.1> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.1.service 2024-08-16T19:03:32.956 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-08-16T19:03:32.956 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-16T19:03:33.214 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:03:33.261 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:33.261 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:33.262 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.262 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.262 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:33.262 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-mon[37413]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:33.262 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:03:33 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:03:33.005+0000 7f07c8afa640 -1 osd.1 0 log_to_monitors true 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:33.337 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:33 smithi161 ceph-mon[37576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:33 smithi062 ceph-mon[32993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-16T19:03:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-mon[37413]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-16T19:03:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-mon[37413]: osdmap e11: 2 total, 1 up, 2 in 2024-08-16T19:03:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-mon[37413]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-mon[37413]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:34.580 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:03:34.369+0000 7f07c4a7d640 -1 osd.1 0 waiting for initial osdmap 2024-08-16T19:03:34.580 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:03:34 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:03:34.372+0000 7f07c08a5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:03:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:34 smithi161 ceph-mon[37576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-16T19:03:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:34 smithi161 ceph-mon[37576]: osdmap e11: 2 total, 1 up, 2 in 2024-08-16T19:03:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:34 smithi161 ceph-mon[37576]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:34 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:34 smithi161 ceph-mon[37576]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:34.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:34 smithi062 ceph-mon[32993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-16T19:03:34.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:34 smithi062 ceph-mon[32993]: osdmap e11: 2 total, 1 up, 2 in 2024-08-16T19:03:34.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:34 smithi062 ceph-mon[32993]: from='osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:34.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:34 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:34.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:34 smithi062 ceph-mon[32993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:03:35.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:35 smithi161 ceph-mon[37576]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:35.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:35 smithi161 ceph-mon[37576]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:03:35.405 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:35 smithi161 ceph-mon[37576]: osdmap e12: 2 total, 1 up, 2 in 2024-08-16T19:03:35.406 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:35.406 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:35.420 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:03:35.446 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-08-16T19:03:35.640 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:35 smithi155 ceph-mon[37413]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:35.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:35 smithi155 ceph-mon[37413]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:03:35.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:35 smithi155 ceph-mon[37413]: osdmap e12: 2 total, 1 up, 2 in 2024-08-16T19:03:35.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:35.641 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:35.713 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:03:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:35 smithi062 ceph-mon[32993]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-16T19:03:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:35 smithi062 ceph-mon[32993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:03:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:35 smithi062 ceph-mon[32993]: osdmap e12: 2 total, 1 up, 2 in 2024-08-16T19:03:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 -- 172.21.15.161:0/375142852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c101a30 msgr2=0x7f883c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 --2- 172.21.15.161:0/375142852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c101a30 0x7f883c101eb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8828009900 tx=0x7f882802a410 comp rx=0 tx=0).stop 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 -- 172.21.15.161:0/375142852 shutdown_connections 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 --2- 172.21.15.161:0/375142852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c1023f0 0x7f883c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 --2- 172.21.15.161:0/375142852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c101a30 0x7f883c101eb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 --2- 172.21.15.161:0/375142852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 0x7f883c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 -- 172.21.15.161:0/375142852 >> 172.21.15.161:0/375142852 conn(0x7f883c0fbfa0 msgr2=0x7f883c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 -- 172.21.15.161:0/375142852 shutdown_connections 2024-08-16T19:03:36.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.169+0000 7f8843186640 1 -- 172.21.15.161:0/375142852 wait complete. 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 Processor -- start 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 -- start start 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 0x7f883c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 0x7f883c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c560 con 0x7f883c101a30 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c3e0 con 0x7f883c100830 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8843186640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f883c10c6e0 con 0x7f883c1023f0 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8841723640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8841723640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:58366/0 (socket says 172.21.15.161:58366) 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8841723640 1 -- 172.21.15.161:0/1571430787 learned_addr learned my addr 172.21.15.161:0/1571430787 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:03:36.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.170+0000 7f8840f22640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 0x7f883c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f883bfff640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 0x7f883c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 -- 172.21.15.161:0/1571430787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 msgr2=0x7f883c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 0x7f883c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 -- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 msgr2=0x7f883c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 0x7f883c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 -- 172.21.15.161:0/1571430787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883c19f970 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f883bfff640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 0x7f883c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8841723640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f883400b970 tx=0x7f883400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834019070 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883c19fc60 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883400cb40 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834007650 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.171+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883c1a0170 con 0x7f883c1023f0 2024-08-16T19:03:36.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.173+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883c1a03a0 con 0x7f883c1023f0 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.173+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f883401c070 con 0x7f883c1023f0 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.174+0000 7f8839ffb640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 0x7f882c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.174+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f88340ccd50 con 0x7f883c1023f0 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.174+0000 7f8840f22640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 0x7f882c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.175+0000 7f8840f22640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 0x7f882c0a4fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8830005fd0 tx=0x7f883000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:36.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.178+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f883409aeb0 con 0x7f883c1023f0 2024-08-16T19:03:36.374 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:36.373+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f883c1a0570 con 0x7f882c0a2b20 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] boot 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: osdmap e13: 2 total, 2 up, 2 in 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:36.422 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:36.423 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:03:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] boot 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: osdmap e13: 2 total, 2 up, 2 in 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: osd.1 [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] boot 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: osdmap e13: 2 total, 2 up, 2 in 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:37.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:37.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi155 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:37.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:37 smithi161 ceph-mon[37576]: osdmap e14: 2 total, 2 up, 2 in 2024-08-16T19:03:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi155 2024-08-16T19:03:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-16T19:03:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:37.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:37.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:37 smithi062 ceph-mon[32993]: osdmap e14: 2 total, 2 up, 2 in 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi155 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 17088M 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:37.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:37 smithi155 ceph-mon[37413]: osdmap e14: 2 total, 2 up, 2 in 2024-08-16T19:03:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/1015772254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]': finished 2024-08-16T19:03:39.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: osdmap e15: 3 total, 2 up, 3 in 2024-08-16T19:03:39.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:39.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:39 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/1345725454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/1015772254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]': finished 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: osdmap e15: 3 total, 2 up, 3 in 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:39 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/1345725454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/1015772254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]: dispatch 2024-08-16T19:03:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42c0d89e-6fb6-4a52-94d1-8198a745128f"}]': finished 2024-08-16T19:03:39.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: osdmap e15: 3 total, 2 up, 3 in 2024-08-16T19:03:39.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:39.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:39 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/1345725454' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:03:41.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:41 smithi161 ceph-mon[37576]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:41 smithi062 ceph-mon[32993]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:41 smithi155 ceph-mon[37413]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:43.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:43 smithi161 ceph-mon[37576]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:43 smithi062 ceph-mon[32993]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:43 smithi155 ceph-mon[37413]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:45.688 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:45 smithi161 ceph-mon[37576]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:45.689 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-16T19:03:45.689 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:45.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:45 smithi062 ceph-mon[32993]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:45.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-16T19:03:45.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:45 smithi155 ceph-mon[37413]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-16T19:03:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:46.586 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:46 smithi161 ceph-mon[37576]: Deploying daemon osd.2 on smithi161 2024-08-16T19:03:46.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:46 smithi062 ceph-mon[32993]: Deploying daemon osd.2 on smithi161 2024-08-16T19:03:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:46 smithi155 ceph-mon[37413]: Deploying daemon osd.2 on smithi161 2024-08-16T19:03:47.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:47 smithi161 ceph-mon[37576]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:47 smithi062 ceph-mon[32993]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:47 smithi155 ceph-mon[37413]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:49.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:49 smithi161 ceph-mon[37576]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:49.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:49 smithi062 ceph-mon[32993]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:49 smithi155 ceph-mon[37413]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:50.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:50.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:50.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:51.602 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:51 smithi161 ceph-mon[37576]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:51.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:51 smithi062 ceph-mon[32993]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:51 smithi155 ceph-mon[37413]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:52.203 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.202+0000 7f8839ffb640 1 -- 172.21.15.161:0/1571430787 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f883c1a0570 con 0x7f882c0a2b20 2024-08-16T19:03:52.203 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.204+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 msgr2=0x7f882c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.204+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 0x7f882c0a4fe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8830005fd0 tx=0x7f883000b410 comp rx=0 tx=0).stop 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.204+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 msgr2=0x7f883c19f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.204+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f883400b970 tx=0x7f883400be30 comp rx=0 tx=0).stop 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 shutdown_connections 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f883c1023f0 0x7f883c19f300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:52.205 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f883c101a30 0x7f883c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:52.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f882c0a2b20 0x7f882c0a4fe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:52.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 --2- 172.21.15.161:0/1571430787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883c100830 0x7f883c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:52.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 >> 172.21.15.161:0/1571430787 conn(0x7f883c0fbfa0 msgr2=0x7f883c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:52.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 shutdown_connections 2024-08-16T19:03:52.206 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:03:52.205+0000 7f8843186640 1 -- 172.21.15.161:0/1571430787 wait complete. 2024-08-16T19:03:53.096 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.2.service 2024-08-16T19:03:53.098 INFO:tasks.cephadm:Deploying osd.3 on smithi062 with /dev/vg_nvme/lv_3... 2024-08-16T19:03:53.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-16T19:03:53.375 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:03:53.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-mon[37576]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:53 smithi062 ceph-mon[32993]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:53.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:53 smithi155 ceph-mon[37413]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:53.847 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:03:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:03:53.621+0000 7fd30bdd3640 -1 osd.2 0 log_to_monitors true 2024-08-16T19:03:54.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:54 smithi161 ceph-mon[37576]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:54.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:54 smithi161 ceph-mon[37576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:54.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:54 smithi062 ceph-mon[32993]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:54.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:54 smithi062 ceph-mon[32993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:54 smithi155 ceph-mon[37413]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:54 smithi155 ceph-mon[37413]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-16T19:03:55.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:55.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-16T19:03:55.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: osdmap e16: 3 total, 2 up, 3 in 2024-08-16T19:03:55.461 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:55.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:55 smithi062 ceph-mon[32993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: osdmap e16: 3 total, 2 up, 3 in 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-mon[37576]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: osdmap e16: 3 total, 2 up, 3 in 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: from='osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:55 smithi155 ceph-mon[37413]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:03:55.904 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:03:55.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-08-16T19:03:56.107 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:03:55.919+0000 7fd307d56640 -1 osd.2 0 waiting for initial osdmap 2024-08-16T19:03:56.107 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:03:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:03:55.924+0000 7fd2ffb76640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:03:56.199 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: osdmap e17: 3 total, 2 up, 3 in 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='osd.2 ' entity='osd.2' 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:56.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:56.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: osdmap e17: 3 total, 2 up, 3 in 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='osd.2 ' entity='osd.2' 2024-08-16T19:03:56.467 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:56.468 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: osdmap e17: 3 total, 2 up, 3 in 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='osd.2 ' entity='osd.2' 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:03:56.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:56.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:56.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.630+0000 7fdc326a8640 1 -- 172.21.15.62:0/3773890022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c102430 msgr2=0x7fdc2c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 --2- 172.21.15.62:0/3773890022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c102430 0x7fdc2c104820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009900 tx=0x7fdc1c02a410 comp rx=0 tx=0).stop 2024-08-16T19:03:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 -- 172.21.15.62:0/3773890022 shutdown_connections 2024-08-16T19:03:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 --2- 172.21.15.62:0/3773890022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c107690 0x7fdc2c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 --2- 172.21.15.62:0/3773890022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 0x7fdc2c107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 --2- 172.21.15.62:0/3773890022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c102430 0x7fdc2c104820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 -- 172.21.15.62:0/3773890022 >> 172.21.15.62:0/3773890022 conn(0x7fdc2c0fbfb0 msgr2=0x7fdc2c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 -- 172.21.15.62:0/3773890022 shutdown_connections 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 -- 172.21.15.62:0/3773890022 wait complete. 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 Processor -- start 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.631+0000 7fdc326a8640 1 -- start start 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c102430 0x7fdc2c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 0x7fdc2c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10d600 con 0x7fdc2c107690 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10d480 con 0x7fdc2c102430 2024-08-16T19:03:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10d780 con 0x7fdc2c104d60 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 0x7fdc2c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54272/0 (socket says 172.21.15.62:54272) 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 -- 172.21.15.62:0/32210880 learned_addr learned my addr 172.21.15.62:0/32210880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc2bfff640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c102430 0x7fdc2c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 -- 172.21.15.62:0/32210880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 msgr2=0x7fdc2c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 0x7fdc2c1003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 -- 172.21.15.62:0/32210880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c102430 msgr2=0x7fdc2c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c102430 0x7fdc2c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 -- 172.21.15.62:0/32210880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c1011e0 con 0x7fdc2c107690 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc30c45640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400cad0 tx=0x7fdc2400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400ad60 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc24005020 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c1a1e80 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400f6b0 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.632+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a2360 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.634+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c0fdaa0 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.635+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fdc24005560 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.635+0000 7fdc297fa640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 0x7fdc140a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.635+0000 7fdc2bfff640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 0x7fdc140a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.635+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fdc240cd940 con 0x7fdc2c107690 2024-08-16T19:03:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.636+0000 7fdc2bfff640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 0x7fdc140a4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c000c00 tx=0x7fdc1c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:03:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.640+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc24014030 con 0x7fdc2c107690 2024-08-16T19:03:56.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:03:56.826+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc2c062360 con 0x7fdc140a2af0 2024-08-16T19:03:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:03:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi161 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] boot 2024-08-16T19:03:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: osdmap e18: 3 total, 3 up, 3 in 2024-08-16T19:03:57.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi161 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-16T19:03:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] boot 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: osdmap e18: 3 total, 3 up, 3 in 2024-08-16T19:03:57.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi161 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 17088M 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:03:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: osd.2 [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] boot 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: osdmap e18: 3 total, 3 up, 3 in 2024-08-16T19:03:57.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:03:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:58 smithi062 ceph-mon[32993]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:58.482 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:58.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:58 smithi155 ceph-mon[37413]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:58.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:58 smithi161 ceph-mon[37576]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:03:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: pgmap v68: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:03:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: osdmap e19: 3 total, 3 up, 3 in 2024-08-16T19:03:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]: dispatch 2024-08-16T19:03:59.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:03:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: pgmap v68: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: osdmap e19: 3 total, 3 up, 3 in 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]: dispatch 2024-08-16T19:03:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:03:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: pgmap v68: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: osdmap e19: 3 total, 3 up, 3 in 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]: dispatch 2024-08-16T19:03:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:03:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:03:59.803 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:59 smithi062 sudo[51413]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:03:59.803 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:59 smithi062 sudo[51413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:03:59.803 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:59 smithi062 sudo[51413]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:03:59.803 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:03:59 smithi062 sudo[51413]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:00.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:00.240+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc24097f60 con 0x7fdc2c107690 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]': finished 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: osdmap e20: 4 total, 3 up, 4 in 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3751335583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: osdmap e21: 4 total, 3 up, 4 in 2024-08-16T19:04:00.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]': finished 2024-08-16T19:04:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: osdmap e20: 4 total, 3 up, 4 in 2024-08-16T19:04:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3751335583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:00.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: osdmap e21: 4 total, 3 up, 4 in 2024-08-16T19:04:00.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.497 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:04:00 smithi062 sudo[51464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:00.497 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:04:00 smithi062 sudo[51464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:00.497 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:04:00 smithi062 sudo[51464]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:00.497 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:04:00 smithi062 sudo[51464]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2667797849' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d4b830f-2036-4a2e-ab45-59ed8a735275"}]': finished 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: osdmap e20: 4 total, 3 up, 4 in 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3751335583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: osdmap e21: 4 total, 3 up, 4 in 2024-08-16T19:04:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:00.745 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:04:00.746 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:00.746 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45949]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:00.746 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45949]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:01.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:01.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:01.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45952]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:01.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:04:00 smithi155 sudo[45952]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:01.271 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:01 smithi161 ceph-mon[37576]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:01.272 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:01 smithi161 ceph-mon[37576]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-16T19:04:01.272 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.272 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.272 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:04:01.272 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:01.272 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46169]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:01.272 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46169]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:01.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 ceph-mon[32993]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:01.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 ceph-mon[32993]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-16T19:04:01.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:01 smithi155 ceph-mon[37413]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:01 smithi155 ceph-mon[37413]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-16T19:04:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:01.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46172]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:01.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:01.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46172]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:01.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:04:01 smithi161 sudo[46172]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 sudo[52204]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:04:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 sudo[52204]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 sudo[52204]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:01 smithi062 sudo[52204]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:02 smithi062 sudo[52577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:02 smithi062 sudo[52577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:02 smithi062 sudo[52577]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:02 smithi062 sudo[52577]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:02.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46175]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:04:02.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:02.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46175]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:02.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46175]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46178]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:02 smithi161 sudo[46178]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:03.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:02 smithi155 sudo[45955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-16T19:04:03.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:02 smithi155 sudo[45955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:03.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:02 smithi155 sudo[45955]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:03.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:02 smithi155 sudo[45955]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.372 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:03.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.373 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 sudo[45958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 sudo[45958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 sudo[45958]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 sudo[45958]: pam_unix(sudo:session): session closed for user root 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:03.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:03.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:04.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:04 smithi062 ceph-mon[32993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:04 smithi155 ceph-mon[37413]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:04.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:04.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:04.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-16T19:04:04.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:04:04.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:04:04.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:04:04.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:04 smithi161 ceph-mon[37576]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-16T19:04:05.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:05 smithi062 ceph-mon[32993]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:05.480 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:05 smithi155 ceph-mon[37413]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:05.539 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:05 smithi161 ceph-mon[37576]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:06.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-16T19:04:06.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:06.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-16T19:04:06.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:06.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-16T19:04:06.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:07.324 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:07 smithi062 ceph-mon[32993]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-08-16T19:04:07.324 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:07 smithi062 ceph-mon[32993]: Deploying daemon osd.3 on smithi062 2024-08-16T19:04:07.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:07 smithi155 ceph-mon[37413]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-08-16T19:04:07.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:07 smithi155 ceph-mon[37413]: Deploying daemon osd.3 on smithi062 2024-08-16T19:04:07.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:07 smithi161 ceph-mon[37576]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-08-16T19:04:07.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:07 smithi161 ceph-mon[37576]: Deploying daemon osd.3 on smithi062 2024-08-16T19:04:09.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:09 smithi062 ceph-mon[32993]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-08-16T19:04:09.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:09 smithi155 ceph-mon[37413]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-08-16T19:04:09.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:09 smithi161 ceph-mon[37576]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2024-08-16T19:04:11.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:11 smithi062 ceph-mon[32993]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-08-16T19:04:11.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:11.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:11.488 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:11 smithi155 ceph-mon[37413]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-08-16T19:04:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:11.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:11 smithi161 ceph-mon[37576]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-08-16T19:04:11.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:11.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:11.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:13.409 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:13 smithi062 ceph-mon[32993]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-08-16T19:04:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:13 smithi155 ceph-mon[37413]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-08-16T19:04:13.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:13 smithi161 ceph-mon[37576]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-08-16T19:04:14.198 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 3 on host 'smithi062' 2024-08-16T19:04:14.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.197+0000 7fdc297fa640 1 -- 172.21.15.62:0/32210880 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc2c062360 con 0x7fdc140a2af0 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 msgr2=0x7fdc140a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 0x7fdc140a4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c000c00 tx=0x7fdc1c031400 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 msgr2=0x7fdc2c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400cad0 tx=0x7fdc2400ced0 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 shutdown_connections 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc2c107690 0x7fdc2c100920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc2c104d60 0x7fdc2c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc140a2af0 0x7fdc140a4fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 --2- 172.21.15.62:0/32210880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc2c102430 0x7fdc2c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 >> 172.21.15.62:0/32210880 conn(0x7fdc2c0fbfb0 msgr2=0x7fdc2c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 shutdown_connections 2024-08-16T19:04:14.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:04:14.200+0000 7fdc326a8640 1 -- 172.21.15.62:0/32210880 wait complete. 2024-08-16T19:04:14.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.743 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.777 DEBUG:teuthology.orchestra.run.smithi062:osd.3> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.3.service 2024-08-16T19:04:14.779 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_3... 2024-08-16T19:04:14.779 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-16T19:04:14.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:14.996 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:04:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:04:14.882+0000 7f75fb73a640 -1 osd.3 0 log_to_monitors true 2024-08-16T19:04:15.043 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:04:15.809 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:15 smithi155 ceph-mon[37413]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-08-16T19:04:15.810 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:15 smithi155 ceph-mon[37413]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-16T19:04:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:15 smithi161 ceph-mon[37576]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-08-16T19:04:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:15 smithi161 ceph-mon[37576]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-16T19:04:15.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:15 smithi062 ceph-mon[32993]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-08-16T19:04:15.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:15 smithi062 ceph-mon[32993]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-16T19:04:16.525 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-mon[32993]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-16T19:04:16.525 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-mon[32993]: osdmap e22: 4 total, 3 up, 4 in 2024-08-16T19:04:16.525 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-mon[32993]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:04:16.525 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:16.525 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-mon[32993]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:04:16.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:16 smithi155 ceph-mon[37413]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-16T19:04:16.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:16 smithi155 ceph-mon[37413]: osdmap e22: 4 total, 3 up, 4 in 2024-08-16T19:04:16.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:16 smithi155 ceph-mon[37413]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:04:16.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:16 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:16.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:16 smithi155 ceph-mon[37413]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:04:16.801 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:16 smithi161 ceph-mon[37576]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-16T19:04:16.802 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:16 smithi161 ceph-mon[37576]: osdmap e22: 4 total, 3 up, 4 in 2024-08-16T19:04:16.802 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:16 smithi161 ceph-mon[37576]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:04:16.802 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:16 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:16.802 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:16 smithi161 ceph-mon[37576]: from='osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:04:16.996 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:04:16.523+0000 7f75f76bd640 -1 osd.3 0 waiting for initial osdmap 2024-08-16T19:04:16.996 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:04:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:04:16.530+0000 7f75f34e5640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:04:17.412 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T19:04:17.438 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-08-16T19:04:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:17 smithi155 ceph-mon[37413]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:17 smithi155 ceph-mon[37413]: osdmap e23: 4 total, 3 up, 4 in 2024-08-16T19:04:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:17 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:17 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:17.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:17 smithi062 ceph-mon[32993]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:17.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:17 smithi062 ceph-mon[32993]: osdmap e23: 4 total, 3 up, 4 in 2024-08-16T19:04:17.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:17 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:17.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:17 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:17.736 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:04:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:17 smithi161 ceph-mon[37576]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:17 smithi161 ceph-mon[37576]: osdmap e23: 4 total, 3 up, 4 in 2024-08-16T19:04:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:17 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:17 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 -- 172.21.15.155:0/3701699299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 msgr2=0x7fd37c101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:18.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 --2- 172.21.15.155:0/3701699299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c101ec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd364009900 tx=0x7fd36402a410 comp rx=0 tx=0).stop 2024-08-16T19:04:18.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 -- 172.21.15.155:0/3701699299 shutdown_connections 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 --2- 172.21.15.155:0/3701699299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 --2- 172.21.15.155:0/3701699299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c101ec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 --2- 172.21.15.155:0/3701699299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 -- 172.21.15.155:0/3701699299 >> 172.21.15.155:0/3701699299 conn(0x7fd37c0fbfd0 msgr2=0x7fd37c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 -- 172.21.15.155:0/3701699299 shutdown_connections 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.237+0000 7fd382025640 1 -- 172.21.15.155:0/3701699299 wait complete. 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.238+0000 7fd382025640 1 Processor -- start 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.238+0000 7fd382025640 1 -- start start 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:18.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c10d5f0 con 0x7fd37c102400 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c10d470 con 0x7fd37c101a40 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c10d770 con 0x7fd37c100840 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:55354/0 (socket says 172.21.15.155:55354) 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 -- 172.21.15.155:0/2101984909 learned_addr learned my addr 172.21.15.155:0/2101984909 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd37bfff640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 -- 172.21.15.155:0/2101984909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 msgr2=0x7fd37c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd37b7fe640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 -- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 msgr2=0x7fd37c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 -- 172.21.15.155:0/2101984909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c19f890 con 0x7fd37c101a40 2024-08-16T19:04:18.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd37bfff640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd37b7fe640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd372ffd640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd364009230 tx=0x7fd364031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36403f030 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c19fb20 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364003ad0 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.239+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364035070 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.240+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c1a0000 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.241+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd37c107c40 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.242+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd364042030 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.242+0000 7fd3797fa640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 0x7fd3680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.243+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fd3640ee4d0 con 0x7fd37c101a40 2024-08-16T19:04:18.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.243+0000 7fd37b7fe640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 0x7fd3680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:18.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.243+0000 7fd37b7fe640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 0x7fd3680a5040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0058c0 tx=0x7fd36c005d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:04:18.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.247+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd3640bc610 con 0x7fd37c101a40 2024-08-16T19:04:18.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:18.428+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd37c062360 con 0x7fd3680a2b80 2024-08-16T19:04:18.689 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] boot 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: osdmap e24: 4 total, 4 up, 4 in 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:18.691 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:18 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] boot 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: osdmap e24: 4 total, 4 up, 4 in 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:18.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:18 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:04:18.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: osd.3 [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] boot 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: osdmap e24: 4 total, 4 up, 4 in 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:18.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:18 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:19 smithi155 ceph-mon[37413]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:19 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi062 2024-08-16T19:04:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:19 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 8544M 2024-08-16T19:04:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:19 smithi155 ceph-mon[37413]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:19 smithi155 ceph-mon[37413]: osdmap e25: 4 total, 4 up, 4 in 2024-08-16T19:04:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:19 smithi161 ceph-mon[37576]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:19 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi062 2024-08-16T19:04:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:19 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 8544M 2024-08-16T19:04:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:19 smithi161 ceph-mon[37576]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:19 smithi161 ceph-mon[37576]: osdmap e25: 4 total, 4 up, 4 in 2024-08-16T19:04:19.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:19 smithi062 ceph-mon[32993]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-08-16T19:04:19.897 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:19 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi062 2024-08-16T19:04:19.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:19 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 8544M 2024-08-16T19:04:19.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:19 smithi062 ceph-mon[32993]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:19.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:19 smithi062 ceph-mon[32993]: osdmap e25: 4 total, 4 up, 4 in 2024-08-16T19:04:20.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:20 smithi155 ceph-mon[37413]: osdmap e26: 4 total, 4 up, 4 in 2024-08-16T19:04:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:20 smithi161 ceph-mon[37576]: osdmap e26: 4 total, 4 up, 4 in 2024-08-16T19:04:20.914 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:20 smithi062 ceph-mon[32993]: osdmap e26: 4 total, 4 up, 4 in 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: osdmap e27: 4 total, 4 up, 4 in 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/2814405906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]': finished 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: osdmap e28: 5 total, 4 up, 5 in 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:21 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/1727754291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: osdmap e27: 4 total, 4 up, 4 in 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/2814405906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]': finished 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: osdmap e28: 5 total, 4 up, 5 in 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:21 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/1727754291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: osdmap e27: 4 total, 4 up, 4 in 2024-08-16T19:04:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/2814405906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]: dispatch 2024-08-16T19:04:21.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "047fb0e0-b0ac-45b7-8cd9-23e62d1f52de"}]': finished 2024-08-16T19:04:21.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: osdmap e28: 5 total, 4 up, 5 in 2024-08-16T19:04:21.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:21.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:21 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/1727754291' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:23 smithi155 ceph-mon[37413]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-16T19:04:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:23 smithi161 ceph-mon[37576]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-16T19:04:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:23 smithi062 ceph-mon[32993]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-16T19:04:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:25 smithi155 ceph-mon[37413]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-08-16T19:04:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:25 smithi161 ceph-mon[37576]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-08-16T19:04:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:25 smithi062 ceph-mon[32993]: pgmap v91: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-08-16T19:04:26.762 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:26 smithi155 ceph-mon[37413]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-16T19:04:26.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:26 smithi161 ceph-mon[37576]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-16T19:04:26.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:26 smithi062 ceph-mon[32993]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-16T19:04:27.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:27 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-16T19:04:27.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:27 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:27.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:27 smithi155 ceph-mon[37413]: Deploying daemon osd.4 on smithi155 2024-08-16T19:04:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:27 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-16T19:04:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:27 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:27 smithi161 ceph-mon[37576]: Deploying daemon osd.4 on smithi155 2024-08-16T19:04:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:27 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-16T19:04:27.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:27 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:27.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:27 smithi062 ceph-mon[32993]: Deploying daemon osd.4 on smithi155 2024-08-16T19:04:28.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:28 smithi161 ceph-mon[37576]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-16T19:04:28.931 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:28 smithi155 ceph-mon[37413]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-16T19:04:28.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:28 smithi062 ceph-mon[32993]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-16T19:04:30.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:30 smithi161 ceph-mon[37576]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:30.925 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:30 smithi155 ceph-mon[37413]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:30.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:30 smithi062 ceph-mon[32993]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:32 smithi161 ceph-mon[37576]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:32.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:32 smithi155 ceph-mon[37413]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:32.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:32.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:32.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:32 smithi062 ceph-mon[32993]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:34.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:34 smithi155 ceph-mon[37413]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:34.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:34 smithi161 ceph-mon[37576]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:34.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:34 smithi062 ceph-mon[32993]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:35.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.039+0000 7fd3797fa640 1 -- 172.21.15.155:0/2101984909 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd37c062360 con 0x7fd3680a2b80 2024-08-16T19:04:35.040 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-08-16T19:04:35.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 msgr2=0x7fd3680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:35.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 0x7fd3680a5040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0058c0 tx=0x7fd36c005d70 comp rx=0 tx=0).stop 2024-08-16T19:04:35.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 msgr2=0x7fd37c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:35.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd364009230 tx=0x7fd364031b60 comp rx=0 tx=0).stop 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 shutdown_connections 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd37c102400 0x7fd37c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd3680a2b80 0x7fd3680a5040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c101a40 0x7fd37c1981b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 --2- 172.21.15.155:0/2101984909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd37c100840 0x7fd37c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 >> 172.21.15.155:0/2101984909 conn(0x7fd37c0fbfd0 msgr2=0x7fd37c104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 shutdown_connections 2024-08-16T19:04:35.043 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:04:35.041+0000 7fd382025640 1 -- 172.21.15.155:0/2101984909 wait complete. 2024-08-16T19:04:35.810 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.4.service 2024-08-16T19:04:35.813 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-08-16T19:04:35.813 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-16T19:04:36.071 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:04:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:36.829 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:04:36 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:04:36.342+0000 7f35e318a640 -1 osd.4 0 log_to_monitors true 2024-08-16T19:04:37.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:37 smithi155 ceph-mon[37413]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:37.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:37 smithi155 ceph-mon[37413]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:37.298 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:37 smithi155 ceph-mon[37413]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:37 smithi161 ceph-mon[37576]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:37 smithi161 ceph-mon[37576]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:37 smithi161 ceph-mon[37576]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:37 smithi062 ceph-mon[32993]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:37 smithi062 ceph-mon[32993]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:37.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:37 smithi062 ceph-mon[32993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-16T19:04:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:38 smithi161 ceph-mon[37576]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-16T19:04:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:38 smithi161 ceph-mon[37576]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:38 smithi161 ceph-mon[37576]: osdmap e29: 5 total, 4 up, 5 in 2024-08-16T19:04:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:38.577 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:38 smithi161 ceph-mon[37576]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-mon[37413]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-16T19:04:38.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-mon[37413]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-mon[37413]: osdmap e29: 5 total, 4 up, 5 in 2024-08-16T19:04:38.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:38.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-mon[37413]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.580 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:04:38.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-08-16T19:04:38.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:38 smithi062 ceph-mon[32993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-16T19:04:38.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:38 smithi062 ceph-mon[32993]: from='osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:38 smithi062 ceph-mon[32993]: osdmap e29: 5 total, 4 up, 5 in 2024-08-16T19:04:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:38 smithi062 ceph-mon[32993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:04:38.858 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:04:39.277 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:04:38.966+0000 7f35df10d640 -1 osd.4 0 waiting for initial osdmap 2024-08-16T19:04:39.277 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:04:38 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:04:38.972+0000 7f35d6f2d640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 -- 172.21.15.161:0/3152402549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8101a40 msgr2=0x7f68c8101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3152402549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8101a40 0x7f68c8101ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009900 tx=0x7f68b002a410 comp rx=0 tx=0).stop 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 -- 172.21.15.161:0/3152402549 shutdown_connections 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3152402549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8102400 0x7f68c8108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3152402549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8101a40 0x7f68c8101ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3152402549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8100840 0x7f68c8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:39.298 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 -- 172.21.15.161:0/3152402549 >> 172.21.15.161:0/3152402549 conn(0x7f68c80fbfd0 msgr2=0x7f68c80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 -- 172.21.15.161:0/3152402549 shutdown_connections 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.295+0000 7f68ccac6640 1 -- 172.21.15.161:0/3152402549 wait complete. 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 Processor -- start 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 -- start start 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8100840 0x7f68c8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8101a40 0x7f68c8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810d5f0 con 0x7f68c8101a40 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810d470 con 0x7f68c8100840 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68ccac6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68c810d770 con 0x7f68c8102400 2024-08-16T19:04:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68c6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68c6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:52458/0 (socket says 172.21.15.161:52458) 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.296+0000 7f68c659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8100840 0x7f68c8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 -- 172.21.15.161:0/3036392579 learned_addr learned my addr 172.21.15.161:0/3036392579 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68bffff640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8101a40 0x7f68c8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 -- 172.21.15.161:0/3036392579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8100840 msgr2=0x7f68c8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8100840 0x7f68c8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 -- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8101a40 msgr2=0x7f68c8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8101a40 0x7f68c8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 -- 172.21.15.161:0/3036392579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c819f870 con 0x7f68c8102400 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68c6d9d640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68c000eb10 tx=0x7f68c000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004b20 con 0x7f68c8102400 2024-08-16T19:04:39.300 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c0004cc0 con 0x7f68c8102400 2024-08-16T19:04:39.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c819fb60 con 0x7f68c8102400 2024-08-16T19:04:39.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0010760 con 0x7f68c8102400 2024-08-16T19:04:39.301 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.297+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81a0050 con 0x7f68c8102400 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.298+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c8107be0 con 0x7f68c8102400 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.299+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68c0005000 con 0x7f68c8102400 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.300+0000 7f68bf7fe640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 0x7f68b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.300+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f68c00ce560 con 0x7f68c8102400 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.301+0000 7f68c659c640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 0x7f68b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:04:39.302 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.301+0000 7f68c659c640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 0x7f68b40a5040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68c8199130 tx=0x7f68b800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:04:39.305 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.305+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68c009c650 con 0x7f68c8102400 2024-08-16T19:04:39.480 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:39.480+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f68c8062360 con 0x7f68b40a2b80 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: osdmap e30: 5 total, 4 up, 5 in 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:39 smithi155 ceph-mon[37413]: from='osd.4 ' entity='osd.4' 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: osdmap e30: 5 total, 4 up, 5 in 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:39 smithi161 ceph-mon[37576]: from='osd.4 ' entity='osd.4' 2024-08-16T19:04:39.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:39.778 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:04:39.778 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: osdmap e30: 5 total, 4 up, 5 in 2024-08-16T19:04:39.778 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.778 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:39.779 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:39 smithi062 ceph-mon[32993]: from='osd.4 ' entity='osd.4' 2024-08-16T19:04:40.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:04:40.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:04:40.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:40.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] boot 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: osdmap e31: 5 total, 5 up, 5 in 2024-08-16T19:04:40.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:40.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] boot 2024-08-16T19:04:40.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: osdmap e31: 5 total, 5 up, 5 in 2024-08-16T19:04:40.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:40.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:04:40.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: osd.4 [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] boot 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: osdmap e31: 5 total, 5 up, 5 in 2024-08-16T19:04:40.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:04:41.534 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:41 smithi161 ceph-mon[37576]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:41.535 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:41 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi155 2024-08-16T19:04:41.535 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:41 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-16T19:04:41.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:41 smithi155 ceph-mon[37413]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:41.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:41 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi155 2024-08-16T19:04:41.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:41 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-16T19:04:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:41 smithi062 ceph-mon[32993]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-16T19:04:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:41 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi155 2024-08-16T19:04:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:41 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 8544M 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: osdmap e32: 5 total, 5 up, 5 in 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/2873775645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]': finished 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: osdmap e33: 6 total, 5 up, 6 in 2024-08-16T19:04:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:42 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: osdmap e32: 5 total, 5 up, 5 in 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/2873775645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]': finished 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: osdmap e33: 6 total, 5 up, 6 in 2024-08-16T19:04:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:42 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:42.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: osdmap e32: 5 total, 5 up, 5 in 2024-08-16T19:04:42.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/2873775645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]: dispatch 2024-08-16T19:04:42.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5fdaca8b-0017-49e5-9731-7c7c13c5b463"}]': finished 2024-08-16T19:04:42.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: osdmap e33: 6 total, 5 up, 6 in 2024-08-16T19:04:42.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:42 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:43.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:43 smithi155 ceph-mon[37413]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:43.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:43 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/3273474047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:43.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:43 smithi161 ceph-mon[37576]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:43.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:43 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/3273474047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:43 smithi062 ceph-mon[32993]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:43 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/3273474047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:04:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:45 smithi155 ceph-mon[37413]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:45.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:45 smithi161 ceph-mon[37576]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:45.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:45 smithi062 ceph-mon[32993]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:47.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:47 smithi161 ceph-mon[37576]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:47 smithi155 ceph-mon[37413]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:47 smithi062 ceph-mon[32993]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:49.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:49 smithi155 ceph-mon[37413]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:49.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-16T19:04:49.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:49.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:49 smithi155 ceph-mon[37413]: Deploying daemon osd.5 on smithi161 2024-08-16T19:04:49.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:49 smithi161 ceph-mon[37576]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:49.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-16T19:04:49.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:49.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:49 smithi161 ceph-mon[37576]: Deploying daemon osd.5 on smithi161 2024-08-16T19:04:49.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:49 smithi062 ceph-mon[32993]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:49.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-16T19:04:49.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:49.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:49 smithi062 ceph-mon[32993]: Deploying daemon osd.5 on smithi161 2024-08-16T19:04:51.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:51 smithi155 ceph-mon[37413]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:51.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:51 smithi161 ceph-mon[37576]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:51.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:51 smithi062 ceph-mon[32993]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:53.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:53 smithi155 ceph-mon[37413]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:53.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:53 smithi161 ceph-mon[37576]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:53 smithi062 ceph-mon[32993]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:54.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:54.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:54.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:54.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:54.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:54.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:55.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:55 smithi161 ceph-mon[37576]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:55 smithi155 ceph-mon[37413]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:55 smithi062 ceph-mon[32993]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:56.770 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.769+0000 7f68bf7fe640 1 -- 172.21.15.161:0/3036392579 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68c8062360 con 0x7f68b40a2b80 2024-08-16T19:04:56.770 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 msgr2=0x7f68b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 0x7f68b40a5040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68c8199130 tx=0x7f68b800b390 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 msgr2=0x7f68c819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f68c000eb10 tx=0x7f68c000c2d0 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 shutdown_connections 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68c8102400 0x7f68c819f1d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68c8101a40 0x7f68c8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68b40a2b80 0x7f68b40a5040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 --2- 172.21.15.161:0/3036392579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68c8100840 0x7f68c8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 >> 172.21.15.161:0/3036392579 conn(0x7f68c80fbfd0 msgr2=0x7f68c8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:04:56.774 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 shutdown_connections 2024-08-16T19:04:56.775 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:04:56.771+0000 7f68ccac6640 1 -- 172.21.15.161:0/3036392579 wait complete. 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.444 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.5.service 2024-08-16T19:04:57.446 INFO:tasks.cephadm:Deploying osd.6 on smithi062 with /dev/vg_nvme/lv_2... 2024-08-16T19:04:57.447 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-16T19:04:57.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:57.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.476 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:04:57.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:04:57.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:04:57.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.477 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:04:57.723 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:04:58.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:58 smithi062 ceph-mon[32993]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:58.170 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:58 smithi062 ceph-mon[32993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:58 smithi155 ceph-mon[37413]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:58 smithi155 ceph-mon[37413]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:58.332 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:58 smithi161 ceph-mon[37576]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:58.332 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:58 smithi161 ceph-mon[37576]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: osdmap e34: 6 total, 5 up, 6 in 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.097 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:04:59 smithi062 ceph-mon[32993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: osdmap e34: 6 total, 5 up, 6 in 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:04:59 smithi155 ceph-mon[37413]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: osdmap e34: 6 total, 5 up, 6 in 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: from='osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-mon[37576]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:04:59.389 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:04:59.028+0000 7f09f865a640 -1 osd.5 0 waiting for initial osdmap 2024-08-16T19:04:59.389 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:04:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:04:59.034+0000 7f09f4482640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:05:00.168 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:00 smithi062 ceph-mon[32993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:05:00.325 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:00 smithi062 ceph-mon[32993]: osdmap e35: 6 total, 5 up, 6 in 2024-08-16T19:05:00.325 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.325 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.326 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:00 smithi161 ceph-mon[37576]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:05:00.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:00 smithi161 ceph-mon[37576]: osdmap e35: 6 total, 5 up, 6 in 2024-08-16T19:05:00.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.327 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:05:00.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-08-16T19:05:00.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:00 smithi155 ceph-mon[37413]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:05:00.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:00 smithi155 ceph-mon[37413]: osdmap e35: 6 total, 5 up, 6 in 2024-08-16T19:05:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:00.645 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:05:01.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 -- 172.21.15.62:0/931384483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050101a00 msgr2=0x7f7050101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:01.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 --2- 172.21.15.62:0/931384483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050101a00 0x7f7050101e80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7040009900 tx=0x7f704002a410 comp rx=0 tx=0).stop 2024-08-16T19:05:01.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 -- 172.21.15.62:0/931384483 shutdown_connections 2024-08-16T19:05:01.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 --2- 172.21.15.62:0/931384483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70501023c0 0x7f7050108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:01.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 --2- 172.21.15.62:0/931384483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7050101a00 0x7f7050101e80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:01.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 --2- 172.21.15.62:0/931384483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050100800 0x7f7050100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:01.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.090+0000 7f7057c12640 1 -- 172.21.15.62:0/931384483 >> 172.21.15.62:0/931384483 conn(0x7f70500fbfb0 msgr2=0x7f70500fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- 172.21.15.62:0/931384483 shutdown_connections 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- 172.21.15.62:0/931384483 wait complete. 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 Processor -- start 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- start start 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7050100800 0x7f705006a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 0x7f705019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:01.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:01.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f705010c460 con 0x7f70501023c0 2024-08-16T19:05:01.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f705010c2e0 con 0x7f7050100800 2024-08-16T19:05:01.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.091+0000 7f7057c12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f705010c5e0 con 0x7f7050101a00 2024-08-16T19:05:01.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:01.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41616/0 (socket says 172.21.15.62:41616) 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 -- 172.21.15.62:0/1259805900 learned_addr learned my addr 172.21.15.62:0/1259805900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70551ad640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 0x7f705019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70559ae640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7050100800 0x7f705006a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 -- 172.21.15.62:0/1259805900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 msgr2=0x7f705019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 0x7f705019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 -- 172.21.15.62:0/1259805900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7050100800 msgr2=0x7f705006a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7050100800 0x7f705006a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:01.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 -- 172.21.15.62:0/1259805900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705019f8a0 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70551ad640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 0x7f705019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f70561af640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f704800eb10 tx=0x7f704800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004b20 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048004cc0 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705019fb30 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048010760 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.092+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705019fff0 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.094+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7050100c00 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.095+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7048005020 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.095+0000 7f7046ffd640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 0x7f703c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.095+0000 7f70559ae640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 0x7f703c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.095+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4562+0+0 (secure 0 0 0) 0x7f70480ce840 con 0x7f70501023c0 2024-08-16T19:05:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.096+0000 7f70559ae640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 0x7f703c0a5040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c005d00 tx=0x7f704c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:01.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.100+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f704809c640 con 0x7f70501023c0 2024-08-16T19:05:01.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:01.285+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7050062420 con 0x7f703c0a2b80 2024-08-16T19:05:01.289 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] boot 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: osdmap e36: 6 total, 6 up, 6 in 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi161 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.290 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-16T19:05:01.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:01.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:01.291 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:01 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] boot 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: osdmap e36: 6 total, 6 up, 6 in 2024-08-16T19:05:01.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi161 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:01.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:01 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: osd.5 [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] boot 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: osdmap e36: 6 total, 6 up, 6 in 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:05:01.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi161 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 8544M 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:01.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:01 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:02.303 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:02 smithi062 ceph-mon[32993]: osdmap e37: 6 total, 6 up, 6 in 2024-08-16T19:05:02.304 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:02 smithi062 ceph-mon[32993]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:02.304 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:02.304 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:02.304 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:02 smithi155 ceph-mon[37413]: osdmap e37: 6 total, 6 up, 6 in 2024-08-16T19:05:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:02 smithi155 ceph-mon[37413]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:02 smithi161 ceph-mon[37576]: osdmap e37: 6 total, 6 up, 6 in 2024-08-16T19:05:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:02 smithi161 ceph-mon[37576]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:03.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:03 smithi062 ceph-mon[32993]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:03.206 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:03 smithi062 ceph-mon[32993]: osdmap e38: 6 total, 6 up, 6 in 2024-08-16T19:05:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:03 smithi155 ceph-mon[37413]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:03 smithi155 ceph-mon[37413]: osdmap e38: 6 total, 6 up, 6 in 2024-08-16T19:05:03.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:03 smithi161 ceph-mon[37576]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:03.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:03 smithi161 ceph-mon[37576]: osdmap e38: 6 total, 6 up, 6 in 2024-08-16T19:05:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:04 smithi155 ceph-mon[37413]: osdmap e39: 6 total, 6 up, 6 in 2024-08-16T19:05:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:04 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]: dispatch 2024-08-16T19:05:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:04 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]': finished 2024-08-16T19:05:04.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:04 smithi155 ceph-mon[37413]: osdmap e40: 7 total, 6 up, 7 in 2024-08-16T19:05:04.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:04 smithi161 ceph-mon[37576]: osdmap e39: 6 total, 6 up, 6 in 2024-08-16T19:05:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:04 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]: dispatch 2024-08-16T19:05:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:04 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]': finished 2024-08-16T19:05:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:04 smithi161 ceph-mon[37576]: osdmap e40: 7 total, 6 up, 7 in 2024-08-16T19:05:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:04 smithi062 ceph-mon[32993]: osdmap e39: 6 total, 6 up, 6 in 2024-08-16T19:05:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:04 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]: dispatch 2024-08-16T19:05:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:04 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2401545950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a966574b-ec4f-4f19-848c-a755ecbda258"}]': finished 2024-08-16T19:05:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:04 smithi062 ceph-mon[32993]: osdmap e40: 7 total, 6 up, 7 in 2024-08-16T19:05:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:05.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:05 smithi155 ceph-mon[37413]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:05.333 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:05 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3399247082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:05.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:05 smithi161 ceph-mon[37576]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:05.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:05 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3399247082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:05.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:05 smithi062 ceph-mon[32993]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:05.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:05 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3399247082' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:07.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:07 smithi155 ceph-mon[37413]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-08-16T19:05:07.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:07 smithi161 ceph-mon[37576]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-08-16T19:05:07.462 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:07 smithi062 ceph-mon[32993]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-08-16T19:05:09.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:09 smithi062 ceph-mon[32993]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:05:09.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:09 smithi155 ceph-mon[37413]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:05:09.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:09 smithi161 ceph-mon[37576]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:05:11.161 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:11 smithi062 ceph-mon[32993]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-16T19:05:11.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-16T19:05:11.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:11.162 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:11 smithi062 ceph-mon[32993]: Deploying daemon osd.6 on smithi062 2024-08-16T19:05:11.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:11 smithi155 ceph-mon[37413]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-16T19:05:11.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-16T19:05:11.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:11.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:11 smithi155 ceph-mon[37413]: Deploying daemon osd.6 on smithi062 2024-08-16T19:05:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:11 smithi161 ceph-mon[37576]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-08-16T19:05:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-16T19:05:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:11 smithi161 ceph-mon[37576]: Deploying daemon osd.6 on smithi062 2024-08-16T19:05:13.237 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:13 smithi062 ceph-mon[32993]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-16T19:05:13.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:13 smithi155 ceph-mon[37413]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-16T19:05:13.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:13 smithi161 ceph-mon[37576]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-08-16T19:05:15.226 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:15 smithi062 ceph-mon[32993]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-08-16T19:05:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:15 smithi155 ceph-mon[37413]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-08-16T19:05:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:15 smithi161 ceph-mon[37576]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-08-16T19:05:16.177 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:16 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:16.177 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:16 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:16.177 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:16 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:16.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:16 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:16.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:16 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:16.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:16 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:16.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:16 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:16 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:16 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:17 smithi062 ceph-mon[32993]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-16T19:05:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:17 smithi155 ceph-mon[37413]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-16T19:05:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:17 smithi161 ceph-mon[37576]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-16T19:05:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.403+0000 7f7046ffd640 1 -- 172.21.15.62:0/1259805900 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7050062420 con 0x7f703c0a2b80 2024-08-16T19:05:18.405 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 6 on host 'smithi062' 2024-08-16T19:05:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 msgr2=0x7f703c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 0x7f703c0a5040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c005d00 tx=0x7f704c005c50 comp rx=0 tx=0).stop 2024-08-16T19:05:18.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 msgr2=0x7f705019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f704800eb10 tx=0x7f704800c2d0 comp rx=0 tx=0).stop 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 shutdown_connections 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70501023c0 0x7f705019ef90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7050101a00 0x7f705019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f703c0a2b80 0x7f703c0a5040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 --2- 172.21.15.62:0/1259805900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7050100800 0x7f705006a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 >> 172.21.15.62:0/1259805900 conn(0x7f70500fbfb0 msgr2=0x7f7050104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 shutdown_connections 2024-08-16T19:05:18.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:05:18.405+0000 7f7057c12640 1 -- 172.21.15.62:0/1259805900 wait complete. 2024-08-16T19:05:19.044 DEBUG:teuthology.orchestra.run.smithi062:osd.6> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.6.service 2024-08-16T19:05:19.047 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_2... 2024-08-16T19:05:19.047 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-16T19:05:19.311 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:05:19.676 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:19 smithi155 ceph-mon[37413]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:19.676 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:19 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.676 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:19 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.676 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:19 smithi155 ceph-mon[37413]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-16T19:05:19.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:19 smithi062 ceph-mon[32993]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:19.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:19 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:19 smithi062 ceph-mon[32993]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-16T19:05:19.747 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:05:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:05:19.388+0000 7ff5154a2640 -1 osd.6 0 log_to_monitors true 2024-08-16T19:05:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:19 smithi161 ceph-mon[37576]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:19 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:19 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:19 smithi161 ceph-mon[37576]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: osdmap e41: 7 total, 6 up, 7 in 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:20.637 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:20 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: osdmap e41: 7 total, 6 up, 7 in 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:20.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:20 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: osdmap e41: 7 total, 6 up, 7 in 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:20 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:21.577 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:21.578 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:05:21.578 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: osdmap e42: 7 total, 6 up, 7 in 2024-08-16T19:05:21.578 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.578 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.578 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-mon[32993]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' 2024-08-16T19:05:21.578 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:05:21.297+0000 7ff511425640 -1 osd.6 0 waiting for initial osdmap 2024-08-16T19:05:21.578 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:05:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:05:21.306+0000 7ff509245640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: osdmap e42: 7 total, 6 up, 7 in 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:21 smithi155 ceph-mon[37413]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' 2024-08-16T19:05:21.814 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T19:05:21.840 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-08-16T19:05:21.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:05:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: osdmap e42: 7 total, 6 up, 7 in 2024-08-16T19:05:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:21 smithi161 ceph-mon[37576]: from='osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322]' entity='osd.6' 2024-08-16T19:05:22.102 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.572+0000 7f1e59ec4640 1 -- 172.21.15.155:0/2048865401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541075f0 msgr2=0x7f1e54109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.572+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/2048865401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541075f0 0x7f1e54109a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009ca0 tx=0x7f1e4c02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- 172.21.15.155:0/2048865401 shutdown_connections 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/2048865401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e541075f0 0x7f1e54109a40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/2048865401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54104cc0 0x7f1e541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/2048865401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e54102390 0x7f1e54104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- 172.21.15.155:0/2048865401 >> 172.21.15.155:0/2048865401 conn(0x7f1e540fbfb0 msgr2=0x7f1e540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- 172.21.15.155:0/2048865401 shutdown_connections 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- 172.21.15.155:0/2048865401 wait complete. 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 Processor -- start 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- start start 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54102390 0x7f1e54199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 0x7f1e541a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410d590 con 0x7f1e541075f0 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410d410 con 0x7f1e54104cc0 2024-08-16T19:05:22.599 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.573+0000 7f1e59ec4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410d710 con 0x7f1e54102390 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:53570/0 (socket says 172.21.15.155:53570) 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 -- 172.21.15.155:0/1631045995 learned_addr learned my addr 172.21.15.155:0/1631045995 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e53fff640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 0x7f1e541a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e537fe640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54102390 0x7f1e54199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 -- 172.21.15.155:0/1631045995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54102390 msgr2=0x7f1e54199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54102390 0x7f1e54199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 -- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 msgr2=0x7f1e541a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 0x7f1e541a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 -- 172.21.15.155:0/1631045995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e541a19c0 con 0x7f1e54104cc0 2024-08-16T19:05:22.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e53fff640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 0x7f1e541a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e52ffd640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800c970 tx=0x7f1e4800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48014070 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48004440 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541a1cb0 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800f450 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.574+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541a21a0 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.575+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e541090e0 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.577+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1e48017070 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.578+0000 7f1e50ff9640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 0x7f1e3c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.578+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f1e480ce5e0 con 0x7f1e54104cc0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.578+0000 7f1e537fe640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 0x7f1e3c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.579+0000 7f1e537fe640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 0x7f1e3c0a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005d00 tx=0x7f1e44005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:22.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.581+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1e4801e030 con 0x7f1e54104cc0 2024-08-16T19:05:22.700 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:05:22.701 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:05:22.701 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:22.701 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] boot 2024-08-16T19:05:22.701 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: osdmap e43: 7 total, 7 up, 7 in 2024-08-16T19:05:22.701 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:22 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:22.756 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:22.755+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1e54062360 con 0x7f1e3c0a2bb0 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] boot 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: osdmap e43: 7 total, 7 up, 7 in 2024-08-16T19:05:22.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:22 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: osd.6 [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] boot 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: osdmap e43: 7 total, 7 up, 7 in 2024-08-16T19:05:22.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:22 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:05:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:23 smithi155 ceph-mon[37413]: osdmap e44: 7 total, 7 up, 7 in 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:23 smithi161 ceph-mon[37576]: osdmap e44: 7 total, 7 up, 7 in 2024-08-16T19:05:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-16T19:05:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:23.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:23.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:23.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:23 smithi062 ceph-mon[32993]: osdmap e44: 7 total, 7 up, 7 in 2024-08-16T19:05:24.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:24 smithi155 ceph-mon[37413]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:24.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:24 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi062 2024-08-16T19:05:24.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:24 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 5696M 2024-08-16T19:05:24.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:24 smithi161 ceph-mon[37576]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:24.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:24 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi062 2024-08-16T19:05:24.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:24 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 5696M 2024-08-16T19:05:24.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:24 smithi062 ceph-mon[32993]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:24.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:24 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi062 2024-08-16T19:05:24.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:24 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 5696M 2024-08-16T19:05:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/1066968250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]': finished 2024-08-16T19:05:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: osdmap e45: 8 total, 7 up, 8 in 2024-08-16T19:05:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:25 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/1066968250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]': finished 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: osdmap e45: 8 total, 7 up, 8 in 2024-08-16T19:05:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:25 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/1066968250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]: dispatch 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61e8cc89-9d50-46d6-8d08-9762b73cdf70"}]': finished 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: osdmap e45: 8 total, 7 up, 8 in 2024-08-16T19:05:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:25 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:26.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:26 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/2010477556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:26.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:26 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/2010477556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:26.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:26 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/2010477556' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:27 smithi155 ceph-mon[37413]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:27 smithi161 ceph-mon[37576]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:27 smithi062 ceph-mon[32993]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:29.814 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:29 smithi155 ceph-mon[37413]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:29 smithi161 ceph-mon[37576]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:29 smithi062 ceph-mon[32993]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:31.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:31 smithi155 ceph-mon[37413]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:31 smithi161 ceph-mon[37576]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:31.902 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:31 smithi062 ceph-mon[32993]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:32.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-16T19:05:32.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:32 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-16T19:05:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:32 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-16T19:05:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:32 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:33.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:33 smithi155 ceph-mon[37413]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:33.784 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:33 smithi155 ceph-mon[37413]: Deploying daemon osd.7 on smithi155 2024-08-16T19:05:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:33 smithi161 ceph-mon[37576]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:33 smithi161 ceph-mon[37576]: Deploying daemon osd.7 on smithi155 2024-08-16T19:05:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:33 smithi062 ceph-mon[32993]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:33 smithi062 ceph-mon[32993]: Deploying daemon osd.7 on smithi155 2024-08-16T19:05:35.811 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:35 smithi155 ceph-mon[37413]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:35 smithi161 ceph-mon[37576]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:35 smithi062 ceph-mon[32993]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:37.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:37 smithi155 ceph-mon[37413]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:37.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:37.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:37.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:37 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:37 smithi161 ceph-mon[37576]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:37 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:37 smithi062 ceph-mon[32993]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:37.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:37.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:37.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:37 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:39.742 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:39 smithi155 ceph-mon[37413]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:39 smithi161 ceph-mon[37576]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:39 smithi062 ceph-mon[32993]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:40.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.798+0000 7f1e50ff9640 1 -- 172.21.15.155:0/1631045995 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1e54062360 con 0x7f1e3c0a2bb0 2024-08-16T19:05:40.800 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 msgr2=0x7f1e3c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 0x7f1e3c0a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e44005d00 tx=0x7f1e44005c50 comp rx=0 tx=0).stop 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 msgr2=0x7f1e5419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800c970 tx=0x7f1e4800ce30 comp rx=0 tx=0).stop 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 shutdown_connections 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e541075f0 0x7f1e541a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:40.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1e3c0a2bb0 0x7f1e3c0a5070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:40.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e54104cc0 0x7f1e5419a340 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:40.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 --2- 172.21.15.155:0/1631045995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e54102390 0x7f1e54199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:40.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 >> 172.21.15.155:0/1631045995 conn(0x7f1e540fbfb0 msgr2=0x7f1e541019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:05:40.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 shutdown_connections 2024-08-16T19:05:40.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:05:40.800+0000 7f1e59ec4640 1 -- 172.21.15.155:0/1631045995 wait complete. 2024-08-16T19:05:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:41 smithi161 ceph-mon[37576]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.415 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:41 smithi155 ceph-mon[37413]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:41.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.443 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.7.service 2024-08-16T19:05:41.446 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-08-16T19:05:41.446 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-16T19:05:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:41 smithi062 ceph-mon[32993]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:41.707 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:05:42.319 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:42 smithi155 ceph-mon[37413]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:42.319 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:42 smithi155 ceph-mon[37413]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:42 smithi161 ceph-mon[37576]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:42 smithi161 ceph-mon[37576]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:42.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:42 smithi062 ceph-mon[32993]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:42.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:42 smithi062 ceph-mon[32993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-16T19:05:43.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:43.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-16T19:05:43.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: osdmap e46: 8 total, 7 up, 8 in 2024-08-16T19:05:43.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:43.360 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:43.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-mon[37413]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:43.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:05:43.111+0000 7fdf65198640 -1 osd.7 0 waiting for initial osdmap 2024-08-16T19:05:43.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:05:43 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:05:43.120+0000 7fdf60fc0640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:05:43.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:43.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-16T19:05:43.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: osdmap e46: 8 total, 7 up, 8 in 2024-08-16T19:05:43.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:43.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:43.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:43 smithi161 ceph-mon[37576]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:43.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:43.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-16T19:05:43.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: osdmap e46: 8 total, 7 up, 8 in 2024-08-16T19:05:43.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:43.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: from='osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:43.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:43 smithi062 ceph-mon[32993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:05:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:44 smithi161 ceph-mon[37576]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:05:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:44 smithi161 ceph-mon[37576]: osdmap e47: 8 total, 7 up, 8 in 2024-08-16T19:05:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:05:44.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-08-16T19:05:44.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:44 smithi155 ceph-mon[37413]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:05:44.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:44 smithi155 ceph-mon[37413]: osdmap e47: 8 total, 7 up, 8 in 2024-08-16T19:05:44.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.457 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:05:44.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:44 smithi062 ceph-mon[32993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:05:44.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:44 smithi062 ceph-mon[32993]: osdmap e47: 8 total, 7 up, 8 in 2024-08-16T19:05:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 -- 172.21.15.161:0/4023610161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a441023c0 msgr2=0x7f4a44108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 --2- 172.21.15.161:0/4023610161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a441023c0 0x7f4a44108c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009ca0 tx=0x7f4a3c02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 -- 172.21.15.161:0/4023610161 shutdown_connections 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 --2- 172.21.15.161:0/4023610161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a441023c0 0x7f4a44108c50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 --2- 172.21.15.161:0/4023610161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44101a00 0x7f4a44101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 --2- 172.21.15.161:0/4023610161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a44100800 0x7f4a44100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.900 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 -- 172.21.15.161:0/4023610161 >> 172.21.15.161:0/4023610161 conn(0x7f4a440fbfb0 msgr2=0x7f4a440fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 -- 172.21.15.161:0/4023610161 shutdown_connections 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 -- 172.21.15.161:0/4023610161 wait complete. 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.886+0000 7f4a4a225640 1 Processor -- start 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 -- start start 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 0x7f4a4406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 0x7f4a4419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4410c460 con 0x7f4a441023c0 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4410c2e0 con 0x7f4a44100800 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a4a225640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4410c5e0 con 0x7f4a44101a00 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:44.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:36946/0 (socket says 172.21.15.161:36946) 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 -- 172.21.15.161:0/3818131388 learned_addr learned my addr 172.21.15.161:0/3818131388 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a437fe640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 0x7f4a4406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a43fff640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 0x7f4a4419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 -- 172.21.15.161:0/3818131388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 msgr2=0x7f4a4406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 0x7f4a4406a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 -- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 msgr2=0x7f4a4419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 0x7f4a4419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 -- 172.21.15.161:0/3818131388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4419f730 con 0x7f4a44101a00 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a43fff640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 0x7f4a4419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a437fe640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 0x7f4a4406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a42ffd640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c920 tx=0x7f4a3800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:44.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.887+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38007c70 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.888+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4419fa20 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.888+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a38007e10 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.888+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38018890 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.888+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4419ffb0 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.889+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a44107b30 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.890+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a38018a70 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.890+0000 7f4a40ff9640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 0x7f4a2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.891+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5430+0+0 (secure 0 0 0) 0x7f4a380ce990 con 0x7f4a44101a00 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.891+0000 7f4a437fe640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 0x7f4a2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.891+0000 7f4a437fe640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 0x7f4a2c0a5040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a4406ad80 tx=0x7f4a3400b6b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:05:44.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:44.898+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4a38014030 con 0x7f4a44101a00 2024-08-16T19:05:45.081 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:05:45.080+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4a44062360 con 0x7f4a2c0a2b80 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] boot 2024-08-16T19:05:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: osdmap e48: 8 total, 8 up, 8 in 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:45.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:45.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:45.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] boot 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: osdmap e48: 8 total, 8 up, 8 in 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:45.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: osd.7 [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] boot 2024-08-16T19:05:45.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: osdmap e48: 8 total, 8 up, 8 in 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:45.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:05:45.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:05:45.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:05:45.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:05:45.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:46.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:46 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi155 2024-08-16T19:05:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:46 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-16T19:05:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:46 smithi161 ceph-mon[37576]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:46 smithi161 ceph-mon[37576]: osdmap e49: 8 total, 8 up, 8 in 2024-08-16T19:05:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:46 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi155 2024-08-16T19:05:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:46 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-16T19:05:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:46 smithi062 ceph-mon[32993]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:46 smithi062 ceph-mon[32993]: osdmap e49: 8 total, 8 up, 8 in 2024-08-16T19:05:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:46 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi155 2024-08-16T19:05:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:46 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 5696M 2024-08-16T19:05:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:46 smithi155 ceph-mon[37413]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:05:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:46 smithi155 ceph-mon[37413]: osdmap e49: 8 total, 8 up, 8 in 2024-08-16T19:05:47.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:47 smithi161 ceph-mon[37576]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:47.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:47 smithi161 ceph-mon[37576]: osdmap e50: 8 total, 8 up, 8 in 2024-08-16T19:05:47.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:47 smithi062 ceph-mon[32993]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:47.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:47 smithi062 ceph-mon[32993]: osdmap e50: 8 total, 8 up, 8 in 2024-08-16T19:05:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:47 smithi155 ceph-mon[37413]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:47.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:47 smithi155 ceph-mon[37413]: osdmap e50: 8 total, 8 up, 8 in 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: osdmap e51: 8 total, 8 up, 8 in 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/720177012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]': finished 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: osdmap e52: 9 total, 8 up, 9 in 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:05:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:48 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/1857166147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:48.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: osdmap e51: 8 total, 8 up, 8 in 2024-08-16T19:05:48.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/720177012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]': finished 2024-08-16T19:05:48.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: osdmap e52: 9 total, 8 up, 9 in 2024-08-16T19:05:48.409 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:05:48.409 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:48 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/1857166147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: osdmap e51: 8 total, 8 up, 8 in 2024-08-16T19:05:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/720177012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]: dispatch 2024-08-16T19:05:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31f59120-2671-4158-bf29-e39406478022"}]': finished 2024-08-16T19:05:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: osdmap e52: 9 total, 8 up, 9 in 2024-08-16T19:05:48.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:05:48.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:48 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/1857166147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:05:49.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:49 smithi062 ceph-mon[32993]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:49 smithi155 ceph-mon[37413]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:49 smithi161 ceph-mon[37576]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:51.543 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:51 smithi161 ceph-mon[37576]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:51.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:51 smithi062 ceph-mon[32993]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:51 smithi155 ceph-mon[37413]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:53.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:53 smithi161 ceph-mon[37576]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:53 smithi062 ceph-mon[32993]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:53 smithi155 ceph-mon[37413]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:54.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-16T19:05:54.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:54 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:54.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-16T19:05:54.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:54 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:54.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-16T19:05:54.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:54 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:05:55.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:55 smithi161 ceph-mon[37576]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:55.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:55 smithi161 ceph-mon[37576]: Deploying daemon osd.8 on smithi161 2024-08-16T19:05:55.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:55 smithi062 ceph-mon[32993]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:55.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:55 smithi062 ceph-mon[32993]: Deploying daemon osd.8 on smithi161 2024-08-16T19:05:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:55 smithi155 ceph-mon[37413]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:55 smithi155 ceph-mon[37413]: Deploying daemon osd.8 on smithi161 2024-08-16T19:05:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:57 smithi161 ceph-mon[37576]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:57.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:57 smithi062 ceph-mon[32993]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:57 smithi155 ceph-mon[37413]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:59.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:05:59 smithi062 ceph-mon[32993]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:05:59 smithi155 ceph-mon[37413]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:05:59.868 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:05:59 smithi161 ceph-mon[37576]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:00.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:00.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:00.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:00.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:01 smithi062 ceph-mon[32993]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:01 smithi155 ceph-mon[37413]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:01 smithi161 ceph-mon[37576]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:02.652 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.650+0000 7f4a40ff9640 1 -- 172.21.15.161:0/3818131388 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a44062360 con 0x7f4a2c0a2b80 2024-08-16T19:06:02.652 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 msgr2=0x7f4a2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 0x7f4a2c0a5040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a4406ad80 tx=0x7f4a3400b6b0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 msgr2=0x7f4a4419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800c920 tx=0x7f4a3800cde0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 shutdown_connections 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a441023c0 0x7f4a4419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a44101a00 0x7f4a4419c820 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a2c0a2b80 0x7f4a2c0a5040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 --2- 172.21.15.161:0/3818131388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a44100800 0x7f4a4406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 >> 172.21.15.161:0/3818131388 conn(0x7f4a440fbfb0 msgr2=0x7f4a44104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:02.655 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 shutdown_connections 2024-08-16T19:06:02.656 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:02.653+0000 7f4a4a225640 1 -- 172.21.15.161:0/3818131388 wait complete. 2024-08-16T19:06:03.292 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.8.service 2024-08-16T19:06:03.294 INFO:tasks.cephadm:Deploying osd.9 on smithi062 with /dev/vg_nvme/lv_1... 2024-08-16T19:06:03.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-16T19:06:03.587 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:06:03.638 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:06:03.366+0000 7fa2729de640 -1 osd.8 0 log_to_monitors true 2024-08-16T19:06:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.709 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:04.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:04.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-16T19:06:04.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:04.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: osdmap e53: 9 total, 8 up, 9 in 2024-08-16T19:06:04.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:04.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:04 smithi062 ceph-mon[32993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:04.847 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:04.847 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-16T19:06:04.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:04.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: osdmap e53: 9 total, 8 up, 9 in 2024-08-16T19:06:04.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:04.848 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:04 smithi161 ceph-mon[37576]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:05.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:05.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-16T19:06:05.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: from='osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:05.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: osdmap e53: 9 total, 8 up, 9 in 2024-08-16T19:06:05.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:05.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:04 smithi155 ceph-mon[37413]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:06:05.638 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:06:05.322+0000 7fa26e961640 -1 osd.8 0 waiting for initial osdmap 2024-08-16T19:06:05.638 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:06:05.332+0000 7fa266781640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:06:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:05 smithi062 ceph-mon[32993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:06:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:05 smithi062 ceph-mon[32993]: osdmap e54: 9 total, 8 up, 9 in 2024-08-16T19:06:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:05.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:05 smithi062 ceph-mon[32993]: from='osd.8 ' entity='osd.8' 2024-08-16T19:06:05.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:05 smithi155 ceph-mon[37413]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:06:05.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:05 smithi155 ceph-mon[37413]: osdmap e54: 9 total, 8 up, 9 in 2024-08-16T19:06:05.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:05.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:05.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:05 smithi155 ceph-mon[37413]: from='osd.8 ' entity='osd.8' 2024-08-16T19:06:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-mon[37576]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:06:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-mon[37576]: osdmap e54: 9 total, 8 up, 9 in 2024-08-16T19:06:06.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:06.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:06.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:05 smithi161 ceph-mon[37576]: from='osd.8 ' entity='osd.8' 2024-08-16T19:06:06.468 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:06:06.504 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-08-16T19:06:06.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:06:06.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: pgmap v168: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] boot 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: osdmap e55: 9 total, 9 up, 9 in 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: osdmap e56: 9 total, 9 up, 9 in 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi161 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-16T19:06:06.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:06.748 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.776 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: pgmap v168: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] boot 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: osdmap e55: 9 total, 9 up, 9 in 2024-08-16T19:06:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: osdmap e56: 9 total, 9 up, 9 in 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi161 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:06.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:06.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:06.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:06.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:06:07.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:06:07.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: pgmap v168: 1 pgs: 1 peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: osd.8 [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] boot 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: osdmap e55: 9 total, 9 up, 9 in 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: osdmap e56: 9 total, 9 up, 9 in 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi161 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 5696M 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:07.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 -- 172.21.15.62:0/2197586270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 msgr2=0x7f38c0108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 --2- 172.21.15.62:0/2197586270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c0108c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009ca0 tx=0x7f38b802a8c0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 -- 172.21.15.62:0/2197586270 shutdown_connections 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 --2- 172.21.15.62:0/2197586270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c0108c50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 --2- 172.21.15.62:0/2197586270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 --2- 172.21.15.62:0/2197586270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 -- 172.21.15.62:0/2197586270 >> 172.21.15.62:0/2197586270 conn(0x7f38c00fbfb0 msgr2=0x7f38c00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 -- 172.21.15.62:0/2197586270 shutdown_connections 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.213+0000 7f38c499f640 1 -- 172.21.15.62:0/2197586270 wait complete. 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 Processor -- start 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 -- start start 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c460 con 0x7f38c01023c0 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c2e0 con 0x7f38c0101a00 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.214+0000 7f38c499f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c5e0 con 0x7f38c0100800 2024-08-16T19:06:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48628/0 (socket says 172.21.15.62:48628) 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 -- 172.21.15.62:0/2278693515 learned_addr learned my addr 172.21.15.62:0/2278693515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38be59c640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bdd9b640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 -- 172.21.15.62:0/2278693515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 msgr2=0x7f38c0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 -- 172.21.15.62:0/2278693515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 msgr2=0x7f38c0198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 -- 172.21.15.62:0/2278693515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c019f800 con 0x7f38c01023c0 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38be59c640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:06:07.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bdd9b640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38bed9d640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38b8031ae0 tx=0x7f38b8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8003b70 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b8003d10 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b802fc40 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c019fa90 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.215+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c019ff80 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.216+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c01a01b0 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.217+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38b8004050 con 0x7f38c01023c0 2024-08-16T19:06:07.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.218+0000 7f38af7fe640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 0x7f38a80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:07.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.218+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5858+0+0 (secure 0 0 0) 0x7f38b80ef8a0 con 0x7f38c01023c0 2024-08-16T19:06:07.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.218+0000 7f38be59c640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 0x7f38a80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:07.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.219+0000 7f38be59c640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 0x7f38a80a5070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38c0101860 tx=0x7f38b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:07.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.223+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f38b80bd190 con 0x7f38c01023c0 2024-08-16T19:06:07.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:07.412+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f38c01a0380 con 0x7f38a80a2bb0 2024-08-16T19:06:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:08 smithi161 ceph-mon[37576]: osdmap e57: 9 total, 9 up, 9 in 2024-08-16T19:06:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:08 smithi161 ceph-mon[37576]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:08 smithi062 ceph-mon[32993]: osdmap e57: 9 total, 9 up, 9 in 2024-08-16T19:06:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:08 smithi062 ceph-mon[32993]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:08.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:08.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:08 smithi155 ceph-mon[37413]: osdmap e57: 9 total, 9 up, 9 in 2024-08-16T19:06:08.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:08 smithi155 ceph-mon[37413]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:08.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:08.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:08.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:09.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:09 smithi062 ceph-mon[32993]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:09.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:09 smithi062 ceph-mon[32993]: osdmap e58: 9 total, 9 up, 9 in 2024-08-16T19:06:09.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:09 smithi161 ceph-mon[37576]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:09.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:09 smithi161 ceph-mon[37576]: osdmap e58: 9 total, 9 up, 9 in 2024-08-16T19:06:09.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:09 smithi155 ceph-mon[37413]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:09.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:09 smithi155 ceph-mon[37413]: osdmap e58: 9 total, 9 up, 9 in 2024-08-16T19:06:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:10 smithi161 ceph-mon[37576]: osdmap e59: 9 total, 9 up, 9 in 2024-08-16T19:06:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:10 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]: dispatch 2024-08-16T19:06:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:10 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]': finished 2024-08-16T19:06:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:10 smithi161 ceph-mon[37576]: osdmap e60: 10 total, 9 up, 10 in 2024-08-16T19:06:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:10 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:10.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:10 smithi062 ceph-mon[32993]: osdmap e59: 9 total, 9 up, 9 in 2024-08-16T19:06:10.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:10 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]: dispatch 2024-08-16T19:06:10.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:10 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]': finished 2024-08-16T19:06:10.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:10 smithi062 ceph-mon[32993]: osdmap e60: 10 total, 9 up, 10 in 2024-08-16T19:06:10.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:10 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:10.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:10 smithi155 ceph-mon[37413]: osdmap e59: 9 total, 9 up, 9 in 2024-08-16T19:06:10.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:10 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]: dispatch 2024-08-16T19:06:10.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:10 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2746858859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3ae18d6-96bf-4b37-9fa6-99129e4edf74"}]': finished 2024-08-16T19:06:10.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:10 smithi155 ceph-mon[37413]: osdmap e60: 10 total, 9 up, 10 in 2024-08-16T19:06:10.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:10 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:11.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:11 smithi062 ceph-mon[32993]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:11.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:11 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3359499125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:11 smithi155 ceph-mon[37413]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:11 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3359499125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:11.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:11 smithi161 ceph-mon[37576]: pgmap v175: 1 pgs: 1 peering; 577 KiB data, 668 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:11.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:11 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3359499125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:13.419 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:13 smithi062 ceph-mon[32993]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:06:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:13 smithi155 ceph-mon[37413]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:06:13.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:13 smithi161 ceph-mon[37576]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:06:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:15 smithi062 ceph-mon[32993]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:06:15.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:15 smithi155 ceph-mon[37413]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:06:15.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:15 smithi161 ceph-mon[37576]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:06:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:17 smithi062 ceph-mon[32993]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:06:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:17 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-16T19:06:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:17 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:17 smithi155 ceph-mon[37413]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:06:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:17 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-16T19:06:17.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:17 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:17.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:17 smithi161 ceph-mon[37576]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:06:17.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:17 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-16T19:06:17.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:17 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:18.433 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:18 smithi062 ceph-mon[32993]: Deploying daemon osd.9 on smithi062 2024-08-16T19:06:18.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:18 smithi155 ceph-mon[37413]: Deploying daemon osd.9 on smithi062 2024-08-16T19:06:18.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:18 smithi161 ceph-mon[37576]: Deploying daemon osd.9 on smithi062 2024-08-16T19:06:19.310 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:19 smithi062 ceph-mon[32993]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-16T19:06:19.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:19 smithi161 ceph-mon[37576]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-16T19:06:19.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:19 smithi155 ceph-mon[37413]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-16T19:06:21.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:21 smithi062 ceph-mon[32993]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-16T19:06:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:21 smithi155 ceph-mon[37413]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-16T19:06:21.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:21 smithi161 ceph-mon[37576]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-16T19:06:23.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:23 smithi062 ceph-mon[32993]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-16T19:06:23.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:23.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:23.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:23.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:23 smithi155 ceph-mon[37413]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-16T19:06:23.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:23.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:23.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:23 smithi161 ceph-mon[37576]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-16T19:06:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:25.345 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:25 smithi062 ceph-mon[32993]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:25.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:25 smithi155 ceph-mon[37413]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:25.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:25 smithi161 ceph-mon[37576]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.083+0000 7f38af7fe640 1 -- 172.21.15.62:0/2278693515 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38c01a0380 con 0x7f38a80a2bb0 2024-08-16T19:06:26.084 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 9 on host 'smithi062' 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 msgr2=0x7f38a80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 0x7f38a80a5070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38c0101860 tx=0x7f38b0005f90 comp rx=0 tx=0).stop 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 msgr2=0x7f38c019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38b8031ae0 tx=0x7f38b8031ee0 comp rx=0 tx=0).stop 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 shutdown_connections 2024-08-16T19:06:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38c01023c0 0x7f38c019f190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f38a80a2bb0 0x7f38a80a5070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38c0101a00 0x7f38c0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 --2- 172.21.15.62:0/2278693515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38c0100800 0x7f38c0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 >> 172.21.15.62:0/2278693515 conn(0x7f38c00fbfb0 msgr2=0x7f38c00fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 shutdown_connections 2024-08-16T19:06:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:06:26.086+0000 7f38c499f640 1 -- 172.21.15.62:0/2278693515 wait complete. 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:26.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:26 smithi161 ceph-mon[37576]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-16T19:06:26.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:26 smithi062 ceph-mon[32993]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-16T19:06:26.822 DEBUG:teuthology.orchestra.run.smithi062:osd.9> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.9.service 2024-08-16T19:06:26.824 INFO:tasks.cephadm:Deploying osd.10 on smithi155 with /dev/vg_nvme/lv_1... 2024-08-16T19:06:26.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-16T19:06:26.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:26.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:26 smithi155 ceph-mon[37413]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-16T19:06:27.090 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:06:27.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:27 smithi155 ceph-mon[37413]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:27.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:27 smithi155 ceph-mon[37413]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-16T19:06:27.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:27 smithi155 ceph-mon[37413]: osdmap e61: 10 total, 9 up, 10 in 2024-08-16T19:06:27.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:27 smithi155 ceph-mon[37413]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:06:27.531 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:27 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:27.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:27 smithi161 ceph-mon[37576]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:27.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:27 smithi161 ceph-mon[37576]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-16T19:06:27.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:27 smithi161 ceph-mon[37576]: osdmap e61: 10 total, 9 up, 10 in 2024-08-16T19:06:27.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:27 smithi161 ceph-mon[37576]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:06:27.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:27 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:27.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:27 smithi062 ceph-mon[32993]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:27.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:27 smithi062 ceph-mon[32993]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-16T19:06:27.741 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:27 smithi062 ceph-mon[32993]: osdmap e61: 10 total, 9 up, 10 in 2024-08-16T19:06:27.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:27 smithi062 ceph-mon[32993]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-08-16T19:06:27.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:27 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.086 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:06:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:06:28.818+0000 7f865ec08640 -1 osd.9 0 waiting for initial osdmap 2024-08-16T19:06:29.087 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:06:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:06:28.831+0000 7f8656a28640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: osdmap e62: 10 total, 9 up, 10 in 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:29 smithi161 ceph-mon[37576]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: osdmap e62: 10 total, 9 up, 10 in 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.451 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:29 smithi155 ceph-mon[37413]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' 2024-08-16T19:06:29.454 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-08-16T19:06:29.480 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: osdmap e62: 10 total, 9 up, 10 in 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:29.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:29 smithi062 ceph-mon[32993]: from='osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557]' entity='osd.9' 2024-08-16T19:06:29.732 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:06:30.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 -- 172.21.15.155:0/2750019942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c101a00 msgr2=0x7f761c101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:30.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 --2- 172.21.15.155:0/2750019942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c101a00 0x7f761c101e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7610009900 tx=0x7f761002a410 comp rx=0 tx=0).stop 2024-08-16T19:06:30.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 -- 172.21.15.155:0/2750019942 shutdown_connections 2024-08-16T19:06:30.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 --2- 172.21.15.155:0/2750019942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c1023c0 0x7f761c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:30.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 --2- 172.21.15.155:0/2750019942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c101a00 0x7f761c101e80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 --2- 172.21.15.155:0/2750019942 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c100800 0x7f761c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 -- 172.21.15.155:0/2750019942 >> 172.21.15.155:0/2750019942 conn(0x7f761c0fbfb0 msgr2=0x7f761c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 -- 172.21.15.155:0/2750019942 shutdown_connections 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.188+0000 7f76229f2640 1 -- 172.21.15.155:0/2750019942 wait complete. 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 Processor -- start 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 -- start start 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c100800 0x7f761c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 0x7f761c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c480 con 0x7f761c101a00 2024-08-16T19:06:30.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c300 con 0x7f761c1023c0 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f76229f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10c600 con 0x7f761c100800 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f7620f8f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f7620f8f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56814/0 (socket says 172.21.15.155:56814) 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f761b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 0x7f761c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f7620f8f640 1 -- 172.21.15.155:0/2733390226 learned_addr learned my addr 172.21.15.155:0/2733390226 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.189+0000 7f761bfff640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c100800 0x7f761c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 -- 172.21.15.155:0/2733390226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c100800 msgr2=0x7f761c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c100800 0x7f761c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 -- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 msgr2=0x7f761c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 0x7f761c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:30.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 -- 172.21.15.155:0/2733390226 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c19f870 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f761b7fe640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 0x7f761c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f7620f8f640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f761400cad0 tx=0x7f761400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761400ae20 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7614005020 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761400f6f0 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c19fb60 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.190+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a00a0 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.191+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c1a0270 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.193+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f76140051d0 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.193+0000 7f76197fa640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 0x7f76040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.194+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7f76140cea70 con 0x7f761c1023c0 2024-08-16T19:06:30.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.194+0000 7f761bfff640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 0x7f76040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:30.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.194+0000 7f761bfff640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 0x7f76040a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f761c199130 tx=0x7f760c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:30.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.198+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7614014030 con 0x7f761c1023c0 2024-08-16T19:06:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:06:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:06:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] boot 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: osdmap e63: 10 total, 10 up, 10 in 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:30.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:30.379+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f761c1a0440 con 0x7f76040a2b80 2024-08-16T19:06:30.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:06:30.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:06:30.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] boot 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: osdmap e63: 10 total, 10 up, 10 in 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:30.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:30.382 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: osd.9 [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] boot 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: osdmap e63: 10 total, 10 up, 10 in 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:30.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:31.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:31.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi062 2024-08-16T19:06:31.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 4272M 2024-08-16T19:06:31.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:31.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:31.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:31.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:31 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi062 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 4272M 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:31 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-08-16T19:06:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi062 2024-08-16T19:06:31.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 4272M 2024-08-16T19:06:31.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:31.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:31.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:31.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:31 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:32.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:32 smithi161 ceph-mon[37576]: osdmap e64: 10 total, 10 up, 10 in 2024-08-16T19:06:32.479 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:32 smithi155 ceph-mon[37413]: osdmap e64: 10 total, 10 up, 10 in 2024-08-16T19:06:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:32 smithi062 ceph-mon[32993]: osdmap e64: 10 total, 10 up, 10 in 2024-08-16T19:06:33.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:33.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/343239028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]': finished 2024-08-16T19:06:33.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: osdmap e65: 11 total, 10 up, 11 in 2024-08-16T19:06:33.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/343239028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]': finished 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: osdmap e65: 11 total, 10 up, 11 in 2024-08-16T19:06:33.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/343239028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]: dispatch 2024-08-16T19:06:33.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b50d6aad-7475-4b81-911e-a3d08459d2c7"}]': finished 2024-08-16T19:06:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: osdmap e65: 11 total, 10 up, 11 in 2024-08-16T19:06:33.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:34.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:34 smithi161 ceph-mon[37576]: from='client.? 172.21.15.155:0/193457851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:34.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:34 smithi062 ceph-mon[32993]: from='client.? 172.21.15.155:0/193457851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:34.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:34 smithi155 ceph-mon[37413]: from='client.? 172.21.15.155:0/193457851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:35.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:35 smithi161 ceph-mon[37576]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:35.480 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:35 smithi155 ceph-mon[37413]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:35.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:35 smithi062 ceph-mon[32993]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:37.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:37 smithi155 ceph-mon[37413]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:37.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:37 smithi161 ceph-mon[37576]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:37.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:37 smithi062 ceph-mon[32993]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:39.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:39 smithi155 ceph-mon[37413]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:39.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:39 smithi161 ceph-mon[37576]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:39.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:39 smithi062 ceph-mon[32993]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:40.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-16T19:06:40.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:40 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:40.444 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-16T19:06:40.444 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:40 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:40.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-16T19:06:40.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:40 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:41 smithi161 ceph-mon[37576]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:41.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:41 smithi161 ceph-mon[37576]: Deploying daemon osd.10 on smithi155 2024-08-16T19:06:41.445 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:41 smithi155 ceph-mon[37413]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:41.445 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:41 smithi155 ceph-mon[37413]: Deploying daemon osd.10 on smithi155 2024-08-16T19:06:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:41 smithi062 ceph-mon[32993]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:41 smithi062 ceph-mon[32993]: Deploying daemon osd.10 on smithi155 2024-08-16T19:06:43.277 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:43 smithi155 ceph-mon[37413]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:43.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:43 smithi161 ceph-mon[37576]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:43.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:43 smithi062 ceph-mon[32993]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:45.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:45 smithi155 ceph-mon[37413]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:45.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:45.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:45.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:45 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:45 smithi161 ceph-mon[37576]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:45 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:45 smithi062 ceph-mon[32993]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:45.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:45 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:47.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:47 smithi161 ceph-mon[37576]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:47.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:47 smithi062 ceph-mon[32993]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:47.515 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:47 smithi155 ceph-mon[37413]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:48.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.484+0000 7f76197fa640 1 -- 172.21.15.155:0/2733390226 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f761c1a0440 con 0x7f76040a2b80 2024-08-16T19:06:48.485 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 10 on host 'smithi155' 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 msgr2=0x7f76040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 0x7f76040a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f761c199130 tx=0x7f760c00b410 comp rx=0 tx=0).stop 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 msgr2=0x7f761c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f761400cad0 tx=0x7f761400cf90 comp rx=0 tx=0).stop 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 shutdown_connections 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76040a2b80 0x7f76040a5040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:48.487 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f761c1023c0 0x7f761c19f1d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:48.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f761c101a00 0x7f761c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:48.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 --2- 172.21.15.155:0/2733390226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f761c100800 0x7f761c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:48.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 >> 172.21.15.155:0/2733390226 conn(0x7f761c0fbfb0 msgr2=0x7f761c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:48.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 shutdown_connections 2024-08-16T19:06:48.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-08-16T19:06:48.486+0000 7f76229f2640 1 -- 172.21.15.155:0/2733390226 wait complete. 2024-08-16T19:06:49.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:48 smithi155 ceph-mon[37413]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:49.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:48 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.131 DEBUG:teuthology.orchestra.run.smithi155:osd.10> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.10.service 2024-08-16T19:06:49.133 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-08-16T19:06:49.133 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-16T19:06:49.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:48 smithi161 ceph-mon[37576]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:49.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:48 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:48 smithi062 ceph-mon[32993]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:49.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:48 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:49.395 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:06:49.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:49 smithi155 ceph-mon[37413]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:49.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:49 smithi155 ceph-mon[37413]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:49 smithi161 ceph-mon[37576]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:49 smithi161 ceph-mon[37576]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:50.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:49 smithi062 ceph-mon[32993]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:50.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:49 smithi062 ceph-mon[32993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-16T19:06:50.886 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:50.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-16T19:06:50.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: osdmap e66: 11 total, 10 up, 11 in 2024-08-16T19:06:50.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:50.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:50.887 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-mon[37413]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:50.887 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:06:50.874+0000 7fdd5a798640 -1 osd.10 0 waiting for initial osdmap 2024-08-16T19:06:50.887 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:06:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:06:50.885+0000 7fdd565c0640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:06:51.065 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:51.065 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-16T19:06:51.065 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: osdmap e66: 11 total, 10 up, 11 in 2024-08-16T19:06:51.065 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:51.066 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:51.066 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:50 smithi161 ceph-mon[37576]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:51.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:51.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-16T19:06:51.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: osdmap e66: 11 total, 10 up, 11 in 2024-08-16T19:06:51.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:51.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: from='osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:51.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:50 smithi062 ceph-mon[32993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-08-16T19:06:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:06:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: osdmap e67: 11 total, 10 up, 11 in 2024-08-16T19:06:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] boot 2024-08-16T19:06:51.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: osdmap e68: 11 total, 11 up, 11 in 2024-08-16T19:06:51.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:51 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:51.945 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-16T19:06:51.971 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: osdmap e67: 11 total, 10 up, 11 in 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] boot 2024-08-16T19:06:52.034 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: osdmap e68: 11 total, 11 up, 11 in 2024-08-16T19:06:52.035 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:51 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-08-16T19:06:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: osdmap e67: 11 total, 10 up, 11 in 2024-08-16T19:06:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: osd.10 [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] boot 2024-08-16T19:06:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: osdmap e68: 11 total, 11 up, 11 in 2024-08-16T19:06:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:51 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:06:52.268 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 -- 172.21.15.161:0/1985824987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 msgr2=0x7f2588108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 --2- 172.21.15.161:0/1985824987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f2588108c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2580009ca0 tx=0x7f258002a8c0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 -- 172.21.15.161:0/1985824987 shutdown_connections 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 --2- 172.21.15.161:0/1985824987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f2588108c90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 --2- 172.21.15.161:0/1985824987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588101a40 0x7f2588101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 --2- 172.21.15.161:0/1985824987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588100840 0x7f2588100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 -- 172.21.15.161:0/1985824987 >> 172.21.15.161:0/1985824987 conn(0x7f25880fbfd0 msgr2=0x7f25880fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 -- 172.21.15.161:0/1985824987 shutdown_connections 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 -- 172.21.15.161:0/1985824987 wait complete. 2024-08-16T19:06:52.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.752+0000 7f258d116640 1 Processor -- start 2024-08-16T19:06:52.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 -- start start 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588100840 0x7f258806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588101a40 0x7f258819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f258810c520 con 0x7f2588100840 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f258810c3a0 con 0x7f2588101a40 2024-08-16T19:06:52.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258d116640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f258810c6a0 con 0x7f2588102400 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54726/0 (socket says 172.21.15.161:54726) 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 -- 172.21.15.161:0/3417614446 learned_addr learned my addr 172.21.15.161:0/3417614446 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f2586d9d640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588100840 0x7f258806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258659c640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588101a40 0x7f258819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 -- 172.21.15.161:0/3417614446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588101a40 msgr2=0x7f258819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588101a40 0x7f258819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 -- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588100840 msgr2=0x7f258806b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588100840 0x7f258806b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 -- 172.21.15.161:0/3417614446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25881a1ae0 con 0x7f2588102400 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.753+0000 7f258759e640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2580005e30 tx=0x7f2580031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.754+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580003b70 con 0x7f2588102400 2024-08-16T19:06:52.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.754+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25881a1d10 con 0x7f2588102400 2024-08-16T19:06:52.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.754+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2580003d10 con 0x7f2588102400 2024-08-16T19:06:52.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.754+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258002fd30 con 0x7f2588102400 2024-08-16T19:06:52.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.754+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25881a22d0 con 0x7f2588102400 2024-08-16T19:06:52.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.755+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2588107be0 con 0x7f2588102400 2024-08-16T19:06:52.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.756+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2580004050 con 0x7f2588102400 2024-08-16T19:06:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.756+0000 7f256ffff640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:06:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.757+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6723+0+0 (secure 0 0 0) 0x7f25800efe30 con 0x7f2588102400 2024-08-16T19:06:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.757+0000 7f2586d9d640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:06:52.758 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.757+0000 7f2586d9d640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 0x7f25700a5070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25881018a0 tx=0x7f257c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:06:52.760 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.760+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f25800bd4e0 con 0x7f2588102400 2024-08-16T19:06:52.936 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:06:52.935+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2588062360 con 0x7f25700a2bb0 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: osdmap e69: 11 total, 11 up, 11 in 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi155 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:52 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: osdmap e69: 11 total, 11 up, 11 in 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi155 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:52 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: osdmap e69: 11 total, 11 up, 11 in 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi155 2024-08-16T19:06:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 4272M 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:06:53.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:52 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:06:54.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:53 smithi161 ceph-mon[37576]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:53 smithi161 ceph-mon[37576]: osdmap e70: 11 total, 11 up, 11 in 2024-08-16T19:06:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:53 smithi062 ceph-mon[32993]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:54.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:53 smithi062 ceph-mon[32993]: osdmap e70: 11 total, 11 up, 11 in 2024-08-16T19:06:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:53 smithi155 ceph-mon[37413]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:06:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-16T19:06:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-16T19:06:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:06:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:53 smithi155 ceph-mon[37413]: osdmap e70: 11 total, 11 up, 11 in 2024-08-16T19:06:55.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:55 smithi161 ceph-mon[37576]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:06:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:55 smithi161 ceph-mon[37576]: osdmap e71: 11 total, 11 up, 11 in 2024-08-16T19:06:55.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:55 smithi062 ceph-mon[32993]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:06:55.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:55 smithi062 ceph-mon[32993]: osdmap e71: 11 total, 11 up, 11 in 2024-08-16T19:06:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:55 smithi155 ceph-mon[37413]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:06:55.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:55 smithi155 ceph-mon[37413]: osdmap e71: 11 total, 11 up, 11 in 2024-08-16T19:06:56.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: osdmap e72: 11 total, 11 up, 11 in 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/2118127253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]': finished 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: osdmap e73: 12 total, 11 up, 12 in 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:06:56.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:56 smithi062 ceph-mon[32993]: from='client.? 172.21.15.161:0/2368103883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: osdmap e72: 11 total, 11 up, 11 in 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/2118127253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]': finished 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: osdmap e73: 12 total, 11 up, 12 in 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:06:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:56 smithi155 ceph-mon[37413]: from='client.? 172.21.15.161:0/2368103883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: osdmap e72: 11 total, 11 up, 11 in 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/2118127253' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]: dispatch 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25c71c88-59d5-4d2e-98dd-85072ddb3c0a"}]': finished 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: osdmap e73: 12 total, 11 up, 12 in 2024-08-16T19:06:56.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:06:56.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:56 smithi161 ceph-mon[37576]: from='client.? 172.21.15.161:0/2368103883' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-16T19:06:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:57 smithi155 ceph-mon[37413]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-08-16T19:06:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:57 smithi161 ceph-mon[37576]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-08-16T19:06:57.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:57 smithi062 ceph-mon[32993]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 165 KiB/s, 0 objects/s recovering 2024-08-16T19:06:59.529 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:06:59 smithi161 ceph-mon[37576]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:06:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:06:59 smithi155 ceph-mon[37413]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:06:59.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:06:59 smithi062 ceph-mon[32993]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-08-16T19:07:01.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:01 smithi161 ceph-mon[37576]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:07:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:01 smithi062 ceph-mon[32993]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:07:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:01 smithi155 ceph-mon[37413]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-16T19:07:02.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-16T19:07:02.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:02.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-16T19:07:02.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:02.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-16T19:07:02.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:03 smithi161 ceph-mon[37576]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:07:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:03 smithi161 ceph-mon[37576]: Deploying daemon osd.11 on smithi161 2024-08-16T19:07:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:03 smithi062 ceph-mon[32993]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:07:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:03 smithi062 ceph-mon[32993]: Deploying daemon osd.11 on smithi161 2024-08-16T19:07:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:03 smithi155 ceph-mon[37413]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-08-16T19:07:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:03 smithi155 ceph-mon[37413]: Deploying daemon osd.11 on smithi161 2024-08-16T19:07:05.631 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:05 smithi155 ceph-mon[37413]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:05.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:05 smithi161 ceph-mon[37576]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:05 smithi062 ceph-mon[32993]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:07.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:07 smithi161 ceph-mon[37576]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:07.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:07 smithi062 ceph-mon[32993]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:07 smithi155 ceph-mon[37413]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:08.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:08.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:08.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:08 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:08.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:08.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:08.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:08 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:08.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:08.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:08.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:08 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:09.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:09 smithi161 ceph-mon[37576]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:09.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:09 smithi062 ceph-mon[32993]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:09 smithi155 ceph-mon[37413]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:11.012 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-08-16T19:07:11.012 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.011+0000 7f256ffff640 1 -- 172.21.15.161:0/3417614446 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2588062360 con 0x7f25700a2bb0 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.013+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 msgr2=0x7f25700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.013+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 0x7f25700a5070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25881018a0 tx=0x7f257c00b390 comp rx=0 tx=0).stop 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.013+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 msgr2=0x7f25881a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.013+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2580005e30 tx=0x7f2580031ee0 comp rx=0 tx=0).stop 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 shutdown_connections 2024-08-16T19:07:11.014 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2588102400 0x7f25881a12f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2588101a40 0x7f258819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 --2- 172.21.15.161:0/3417614446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2588100840 0x7f258806b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 >> 172.21.15.161:0/3417614446 conn(0x7f25880fbfd0 msgr2=0x7f2588104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 shutdown_connections 2024-08-16T19:07:11.015 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:11.014+0000 7f258d116640 1 -- 172.21.15.161:0/3417614446 wait complete. 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:11.574 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:11.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.575 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:11 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:11 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.718 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.11.service 2024-08-16T19:07:11.721 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-16T19:07:11.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-16T19:07:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:11.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:11.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:11.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:11 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:12.000 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:12.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:12 smithi062 ceph-mon[32993]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.388 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:12 smithi062 ceph-mon[32993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 -- 172.21.15.62:0/3925495065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 msgr2=0x7fcfc4108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:12.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/3925495065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4108c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009ca0 tx=0x7fcfbc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 -- 172.21.15.62:0/3925495065 shutdown_connections 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/3925495065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4108c50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/3925495065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4101a00 0x7fcfc4101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/3925495065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4100800 0x7fcfc4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 -- 172.21.15.62:0/3925495065 >> 172.21.15.62:0/3925495065 conn(0x7fcfc40fbfb0 msgr2=0x7fcfc40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 -- 172.21.15.62:0/3925495065 shutdown_connections 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.472+0000 7fcfc99e9640 1 -- 172.21.15.62:0/3925495065 wait complete. 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 Processor -- start 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 -- start start 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4100800 0x7fcfc4104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 0x7fcfc4102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:12.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc410c460 con 0x7fcfc41023c0 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc410c2e0 con 0x7fcfc4101a00 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc99e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc410c5e0 con 0x7fcfc4100800 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4100800 0x7fcfc4104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39292/0 (socket says 172.21.15.62:39292) 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.473+0000 7fcfc37fe640 1 -- 172.21.15.62:0/2356910408 learned_addr learned my addr 172.21.15.62:0/2356910408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfbbfff640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 0x7fcfc4102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 -- 172.21.15.62:0/2356910408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4100800 msgr2=0x7fcfc4104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4100800 0x7fcfc4104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 -- 172.21.15.62:0/2356910408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 msgr2=0x7fcfc4102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:12.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 0x7fcfc4102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 -- 172.21.15.62:0/2356910408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4103be0 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfbbfff640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 0x7fcfc4102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc37fe640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc031ae0 tx=0x7fcfbc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc003c30 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc003dd0 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc02fcb0 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc41a1d00 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.474+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc41a21c0 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.475+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc41a2390 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.476+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcfbc004050 con 0x7fcfc41023c0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.478+0000 7fcfc89e7640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 0x7fcfb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.478+0000 7fcfc2ffd640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 0x7fcfb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:12.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.478+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6831+0+0 (secure 0 0 0) 0x7fcfbc0efd60 con 0x7fcfc41023c0 2024-08-16T19:07:12.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.478+0000 7fcfc2ffd640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 0x7fcfb00a50c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4101860 tx=0x7fcfb400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:12.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.482+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcfbc0bd310 con 0x7fcfc41023c0 2024-08-16T19:07:12.565 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:12 smithi161 ceph-mon[37576]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.565 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:12 smithi161 ceph-mon[37576]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:12 smithi155 ceph-mon[37413]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:12 smithi155 ceph-mon[37413]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-16T19:07:12.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.661+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fcfc41a26f0 con 0x7fcfc41023c0 2024-08-16T19:07:12.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.662+0000 7fcfc89e7640 1 -- 172.21.15.62:0/2356910408 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fcfbc0ef020 con 0x7fcfc41023c0 2024-08-16T19:07:12.662 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 msgr2=0x7fcfb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 0x7fcfb00a50c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4101860 tx=0x7fcfb400b390 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 msgr2=0x7fcfc4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc031ae0 tx=0x7fcfbc031ee0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 shutdown_connections 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfc41023c0 0x7fcfc4103320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcfb00a2c00 0x7fcfb00a50c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfc4101a00 0x7fcfc4102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 --2- 172.21.15.62:0/2356910408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4100800 0x7fcfc4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 >> 172.21.15.62:0/2356910408 conn(0x7fcfc40fbfb0 msgr2=0x7fcfc40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 shutdown_connections 2024-08-16T19:07:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:12.664+0000 7fcfc99e9640 1 -- 172.21.15.62:0/2356910408 wait complete. 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: osdmap e74: 12 total, 11 up, 12 in 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:13 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2356910408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:13.478 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1723835211,"num_in_osds":12,"osd_in_since":1723835215,"num_remapped_pgs":0} 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: osdmap e74: 12 total, 11 up, 12 in 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:13 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2356910408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:13.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:13.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-16T19:07:13.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: from='osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: osdmap e74: 12 total, 11 up, 12 in 2024-08-16T19:07:13.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:13.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-08-16T19:07:13.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2356910408' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:13.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:07:13.326+0000 7f330cabd640 -1 osd.11 0 waiting for initial osdmap 2024-08-16T19:07:13.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:07:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:07:13.342+0000 7f33088e5640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-16T19:07:14.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:14 smithi161 ceph-mon[37576]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:07:14.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:14 smithi161 ceph-mon[37576]: osdmap e75: 12 total, 11 up, 12 in 2024-08-16T19:07:14.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:14.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:14 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:14.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-16T19:07:14.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:14 smithi062 ceph-mon[32993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:07:14.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:14 smithi062 ceph-mon[32993]: osdmap e75: 12 total, 11 up, 12 in 2024-08-16T19:07:14.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:14.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:14 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:14.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:14 smithi155 ceph-mon[37413]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-08-16T19:07:14.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:14 smithi155 ceph-mon[37413]: osdmap e75: 12 total, 11 up, 12 in 2024-08-16T19:07:14.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:14.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:14 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 -- 172.21.15.62:0/2165615210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f01023d0 msgr2=0x7f20f01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 --2- 172.21.15.62:0/2165615210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f01023d0 0x7f20f01047c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ca0 tx=0x7f20e002a8c0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 -- 172.21.15.62:0/2165615210 shutdown_connections 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 --2- 172.21.15.62:0/2165615210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0107630 0x7f20f0109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 --2- 172.21.15.62:0/2165615210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f0104d00 0x7f20f01070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 --2- 172.21.15.62:0/2165615210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f01023d0 0x7f20f01047c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 -- 172.21.15.62:0/2165615210 >> 172.21.15.62:0/2165615210 conn(0x7f20f00fbfd0 msgr2=0x7f20f00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 -- 172.21.15.62:0/2165615210 shutdown_connections 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.230+0000 7f20f6168640 1 -- 172.21.15.62:0/2165615210 wait complete. 2024-08-16T19:07:15.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 Processor -- start 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 -- start start 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f01023d0 0x7f20f0197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0104d00 0x7f20f0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d650 con 0x7f20f0107630 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d4d0 con 0x7f20f0104d00 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20f6168640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010d7d0 con 0x7f20f01023d0 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39314/0 (socket says 172.21.15.62:39314) 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 -- 172.21.15.62:0/2509194449 learned_addr learned my addr 172.21.15.62:0/2509194449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20eeffd640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0104d00 0x7f20f0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20ef7fe640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f01023d0 0x7f20f0197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 -- 172.21.15.62:0/2509194449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f01023d0 msgr2=0x7f20f0197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f01023d0 0x7f20f0197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 -- 172.21.15.62:0/2509194449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0104d00 msgr2=0x7f20f0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0104d00 0x7f20f0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.231+0000 7f20effff640 1 -- 172.21.15.62:0/2509194449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f019f950 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20effff640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c970 tx=0x7f20e800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e800ae60 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8005500 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e800f450 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f019fc40 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.232+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a0180 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.233+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f00fdac0 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.234+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f20e8004ee0 con 0x7f20f0107630 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.235+0000 7f20ecff9640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 0x7f20d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:15.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.235+0000 7f20ef7fe640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 0x7f20d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:15.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.235+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f20e80ce730 con 0x7f20f0107630 2024-08-16T19:07:15.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.235+0000 7f20ef7fe640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 0x7f20d80a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e0000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:15.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.239+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f20e8014030 con 0x7f20f0107630 2024-08-16T19:07:15.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.419+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f20f002d790 con 0x7f20f0107630 2024-08-16T19:07:15.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: purged_snaps scrub starts 2024-08-16T19:07:15.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: purged_snaps scrub ok 2024-08-16T19:07:15.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:15.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.420 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] boot 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: osdmap e76: 12 total, 12 up, 12 in 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: Detected new or changed devices on smithi161 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.421 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:15.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:15.422 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:15 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.420+0000 7f20ecff9640 1 -- 172.21.15.62:0/2509194449 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f20e809bb30 con 0x7f20f0107630 2024-08-16T19:07:15.422 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 msgr2=0x7f20d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 0x7f20d80a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e0000f40 comp rx=0 tx=0).stop 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 msgr2=0x7f20f019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c970 tx=0x7f20e800cd70 comp rx=0 tx=0).stop 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 shutdown_connections 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f0107630 0x7f20f019f2b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f20d80a2c00 0x7f20d80a50c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f0104d00 0x7f20f0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 --2- 172.21.15.62:0/2509194449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20f01023d0 0x7f20f0197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 >> 172.21.15.62:0/2509194449 conn(0x7f20f00fbfd0 msgr2=0x7f20f0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 shutdown_connections 2024-08-16T19:07:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:15.422+0000 7f20f6168640 1 -- 172.21.15.62:0/2509194449 wait complete. 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: purged_snaps scrub starts 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: purged_snaps scrub ok 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] boot 2024-08-16T19:07:15.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: osdmap e76: 12 total, 12 up, 12 in 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: Detected new or changed devices on smithi161 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:15.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:15 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: purged_snaps scrub starts 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: purged_snaps scrub ok 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: osd.11 [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] boot 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: osdmap e76: 12 total, 12 up, 12 in 2024-08-16T19:07:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: Detected new or changed devices on smithi161 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 4272M 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:15.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:15 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:16.246 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1723835234,"num_in_osds":12,"osd_in_since":1723835215,"num_remapped_pgs":0} 2024-08-16T19:07:16.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:07:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:16 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2509194449' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:16 smithi062 ceph-mon[32993]: osdmap e77: 12 total, 12 up, 12 in 2024-08-16T19:07:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:16.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:16 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2509194449' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:16.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:16 smithi161 ceph-mon[37576]: osdmap e77: 12 total, 12 up, 12 in 2024-08-16T19:07:16.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:16 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2509194449' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-16T19:07:16.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:16 smithi155 ceph-mon[37413]: osdmap e77: 12 total, 12 up, 12 in 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 -- 172.21.15.62:0/2094110653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c103380 msgr2=0x7f8e2c10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 --2- 172.21.15.62:0/2094110653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c103380 0x7f8e2c10b870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002a480 comp rx=0 tx=0).stop 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 -- 172.21.15.62:0/2094110653 shutdown_connections 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 --2- 172.21.15.62:0/2094110653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c10be90 0x7f8e2c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 --2- 172.21.15.62:0/2094110653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c103380 0x7f8e2c10b870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 --2- 172.21.15.62:0/2094110653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 0x7f8e2c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 -- 172.21.15.62:0/2094110653 >> 172.21.15.62:0/2094110653 conn(0x7f8e2c0fc4c0 msgr2=0x7f8e2c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 -- 172.21.15.62:0/2094110653 shutdown_connections 2024-08-16T19:07:16.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.971+0000 7f8e323df640 1 -- 172.21.15.62:0/2094110653 wait complete. 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 Processor -- start 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 -- start start 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 0x7f8e2c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c103380 0x7f8e2c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c111c90 con 0x7f8e2c10be90 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c111b10 con 0x7f8e2c102a60 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e323df640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c111e10 con 0x7f8e2c103380 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e3097c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e3097c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39334/0 (socket says 172.21.15.62:39334) 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e3097c640 1 -- 172.21.15.62:0/2396727484 learned_addr learned my addr 172.21.15.62:0/2396727484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e2bfff640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 0x7f8e2c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:16.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.972+0000 7f8e2b7fe640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c103380 0x7f8e2c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 -- 172.21.15.62:0/2396727484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c103380 msgr2=0x7f8e2c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c103380 0x7f8e2c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 -- 172.21.15.62:0/2396727484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 msgr2=0x7f8e2c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 0x7f8e2c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 -- 172.21.15.62:0/2396727484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c19d0a0 con 0x7f8e2c10be90 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e3097c640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400eaa0 tx=0x7f8e2400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24004d20 con 0x7f8e2c10be90 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e24005020 con 0x7f8e2c10be90 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c19d390 con 0x7f8e2c10be90 2024-08-16T19:07:16.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240116d0 con 0x7f8e2c10be90 2024-08-16T19:07:16.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.973+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c1a4c70 con 0x7f8e2c10be90 2024-08-16T19:07:16.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.974+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c102e40 con 0x7f8e2c10be90 2024-08-16T19:07:16.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.975+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e240051d0 con 0x7f8e2c10be90 2024-08-16T19:07:16.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.976+0000 7f8e297fa640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 0x7f8e140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:16.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.976+0000 7f8e2bfff640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 0x7f8e140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:16.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.976+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f8e240cf0b0 con 0x7f8e2c10be90 2024-08-16T19:07:16.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.977+0000 7f8e2bfff640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 0x7f8e140a50c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00abe0 tx=0x7f8e1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:16.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:16.979+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8e24016030 con 0x7f8e2c10be90 2024-08-16T19:07:17.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.155+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e2c199450 con 0x7f8e2c10be90 2024-08-16T19:07:17.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.156+0000 7f8e297fa640 1 -- 172.21.15.62:0/2396727484 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18432 (secure 0 0 0) 0x7f8e2c199450 con 0x7f8e2c10be90 2024-08-16T19:07:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:17.157 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":77,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:07:15.513951+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 msgr2=0x7f8e140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 0x7f8e140a50c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00abe0 tx=0x7f8e1c00a2d0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 msgr2=0x7f8e2c19c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400eaa0 tx=0x7f8e2400eea0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 shutdown_connections 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e2c10be90 0x7f8e2c19c9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e2c103380 0x7f8e2c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8e140a2c00 0x7f8e140a50c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 --2- 172.21.15.62:0/2396727484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e2c102a60 0x7f8e2c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:17.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.158+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 >> 172.21.15.62:0/2396727484 conn(0x7f8e2c0fc4c0 msgr2=0x7f8e2c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:17.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.159+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 shutdown_connections 2024-08-16T19:07:17.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:17.159+0000 7f8e323df640 1 -- 172.21.15.62:0/2396727484 wait complete. 2024-08-16T19:07:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:17 smithi155 ceph-mon[37413]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:17 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2396727484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:07:17.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:17 smithi062 ceph-mon[32993]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:17.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:17 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2396727484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:07:17.888 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-16T19:03:58.075413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-16T19:07:17.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-16T19:07:17.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:17 smithi161 ceph-mon[37576]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:17.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:17 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2396727484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:07:18.154 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 -- 172.21.15.62:0/2556216931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 msgr2=0x7f886c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2556216931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c102e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f885c009900 tx=0x7f885c02a480 comp rx=0 tx=0).stop 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 -- 172.21.15.62:0/2556216931 shutdown_connections 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2556216931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c10bea0 0x7f886c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2556216931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c103390 0x7f886c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2556216931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c102e50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 -- 172.21.15.62:0/2556216931 >> 172.21.15.62:0/2556216931 conn(0x7f886c0fc4d0 msgr2=0x7f886c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:18.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 -- 172.21.15.62:0/2556216931 shutdown_connections 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 -- 172.21.15.62:0/2556216931 wait complete. 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.742+0000 7f8871b6a640 1 Processor -- start 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 -- start start 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c103390 0x7f886c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 0x7f886c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f886c111d60 con 0x7f886c102a70 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f886c111be0 con 0x7f886c10bea0 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f8871b6a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f886c111ee0 con 0x7f886c103390 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39362/0 (socket says 172.21.15.62:39362) 2024-08-16T19:07:18.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 -- 172.21.15.62:0/2657389424 learned_addr learned my addr 172.21.15.62:0/2657389424 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886affd640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c103390 0x7f886c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886bfff640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 0x7f886c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 -- 172.21.15.62:0/2657389424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c103390 msgr2=0x7f886c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c103390 0x7f886c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 -- 172.21.15.62:0/2657389424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 msgr2=0x7f886c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 0x7f886c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886b7fe640 1 -- 172.21.15.62:0/2657389424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c19d2f0 con 0x7f886c102a70 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.743+0000 7f886bfff640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 0x7f886c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f886b7fe640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f885c005ab0 tx=0x7f885c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c03f030 con 0x7f886c102a70 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f885c003ce0 con 0x7f886c102a70 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c19d580 con 0x7f886c102a70 2024-08-16T19:07:18.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c035070 con 0x7f886c102a70 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.744+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1a4ec0 con 0x7f886c102a70 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.745+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c102ed0 con 0x7f886c102a70 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.746+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f885c042030 con 0x7f886c102a70 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.747+0000 7f8868ff9640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 0x7f88540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.747+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f885c0ef4e0 con 0x7f886c102a70 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.747+0000 7f886affd640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 0x7f88540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.747+0000 7f886affd640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 0x7f88540a50c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f886c0feaf0 tx=0x7f8860008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:18.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.751+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f885c0bc8e0 con 0x7f886c102a70 2024-08-16T19:07:18.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.926+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f886c106f40 con 0x7f886c102a70 2024-08-16T19:07:18.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.927+0000 7f8868ff9640 1 -- 172.21.15.62:0/2657389424 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f885c0dae20 con 0x7f886c102a70 2024-08-16T19:07:18.927 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.929+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 msgr2=0x7f88540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.929+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 0x7f88540a50c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f886c0feaf0 tx=0x7f8860008040 comp rx=0 tx=0).stop 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.929+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 msgr2=0x7f886c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f885c005ab0 tx=0x7f885c0042f0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 shutdown_connections 2024-08-16T19:07:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f88540a2c00 0x7f88540a50c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f886c10bea0 0x7f886c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f886c103390 0x7f886c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 --2- 172.21.15.62:0/2657389424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c102a70 0x7f886c198340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 >> 172.21.15.62:0/2657389424 conn(0x7f886c0fc4d0 msgr2=0x7f886c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 shutdown_connections 2024-08-16T19:07:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:18.930+0000 7f8871b6a640 1 -- 172.21.15.62:0/2657389424 wait complete. 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.a on smithi062 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.d on smithi062 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.g on smithi062 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.j on smithi062 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.b on smithi155 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.e on smithi155 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.h on smithi155 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.k on smithi155 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-08-16T19:07:19.538 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-08-16T19:07:19.539 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-08-16T19:07:19.539 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-08-16T19:07:19.539 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-08-16T19:07:19.805 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:07:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:19 smithi155 ceph-mon[37413]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:19 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2657389424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:07:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:19 smithi161 ceph-mon[37576]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:19 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2657389424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:07:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:19 smithi062 ceph-mon[32993]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:19.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:19 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2657389424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 -- 172.21.15.161:0/3286855345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610101a40 msgr2=0x7f4610101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 --2- 172.21.15.161:0/3286855345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610101a40 0x7f4610101ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4604009900 tx=0x7f460402a410 comp rx=0 tx=0).stop 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 -- 172.21.15.161:0/3286855345 shutdown_connections 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 --2- 172.21.15.161:0/3286855345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610102400 0x7f4610108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 --2- 172.21.15.161:0/3286855345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610101a40 0x7f4610101ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 --2- 172.21.15.161:0/3286855345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610100840 0x7f4610100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.266+0000 7f4615a74640 1 -- 172.21.15.161:0/3286855345 >> 172.21.15.161:0/3286855345 conn(0x7f46100fbfd0 msgr2=0x7f46100fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- 172.21.15.161:0/3286855345 shutdown_connections 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- 172.21.15.161:0/3286855345 wait complete. 2024-08-16T19:07:20.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 Processor -- start 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- start start 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 0x7f461006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101a40 0x7f461019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f461010c4c0 con 0x7f4610101a40 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f461010c340 con 0x7f4610100840 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.267+0000 7f4615a74640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f461010c640 con 0x7f4610102400 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55272/0 (socket says 172.21.15.161:55272) 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 -- 172.21.15.161:0/882914010 learned_addr learned my addr 172.21.15.161:0/882914010 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460effd640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101a40 0x7f461019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 -- 172.21.15.161:0/882914010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 msgr2=0x7f461006b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460f7fe640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 0x7f461006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:20.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 0x7f461006b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 -- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101a40 msgr2=0x7f461019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101a40 0x7f461019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 -- 172.21.15.161:0/882914010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461019f8d0 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460f7fe640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 0x7f461006b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460ffff640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f460800eaa0 tx=0x7f460800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460800cdb0 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4608005020 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080116b0 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461019fb60 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.268+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a0070 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.269+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4610107c40 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.270+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4608005560 con 0x7f4610102400 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.271+0000 7f460cff9640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 0x7f45f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:20.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.271+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f46080cf1a0 con 0x7f4610102400 2024-08-16T19:07:20.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.271+0000 7f460f7fe640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 0x7f45f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:20.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.272+0000 7f460f7fe640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 0x7f45f80a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46101018a0 tx=0x7f4600005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:20.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.274+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4608016030 con 0x7f4610102400 2024-08-16T19:07:20.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.450+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) -- 0x7f4610062520 con 0x7f45f80a2b80 2024-08-16T19:07:20.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.458+0000 7f460cff9640 1 -- 172.21.15.161:0/882914010 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4610062520 con 0x7f45f80a2b80 2024-08-16T19:07:20.459 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 msgr2=0x7f45f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 0x7f45f80a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f46101018a0 tx=0x7f4600005d30 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 msgr2=0x7f461019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f460800eaa0 tx=0x7f460800ef60 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 shutdown_connections 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4610102400 0x7f461019efc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4610101a40 0x7f461019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45f80a2b80 0x7f45f80a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 --2- 172.21.15.161:0/882914010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4610100840 0x7f461006b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 >> 172.21.15.161:0/882914010 conn(0x7f46100fbfd0 msgr2=0x7f4610104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 shutdown_connections 2024-08-16T19:07:20.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-16T19:07:20.460+0000 7f4615a74640 1 -- 172.21.15.161:0/882914010 wait complete. 2024-08-16T19:07:21.167 DEBUG:teuthology.orchestra.run.smithi062:mds.a> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.a.service 2024-08-16T19:07:21.170 DEBUG:teuthology.orchestra.run.smithi062:mds.d> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.d.service 2024-08-16T19:07:21.213 DEBUG:teuthology.orchestra.run.smithi062:mds.g> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.g.service 2024-08-16T19:07:21.256 DEBUG:teuthology.orchestra.run.smithi062:mds.j> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.j.service 2024-08-16T19:07:21.300 DEBUG:teuthology.orchestra.run.smithi155:mds.b> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.b.service 2024-08-16T19:07:21.303 DEBUG:teuthology.orchestra.run.smithi155:mds.e> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.e.service 2024-08-16T19:07:21.346 DEBUG:teuthology.orchestra.run.smithi155:mds.h> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.h.service 2024-08-16T19:07:21.388 DEBUG:teuthology.orchestra.run.smithi155:mds.k> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.k.service 2024-08-16T19:07:21.431 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.c.service 2024-08-16T19:07:21.434 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.f.service 2024-08-16T19:07:21.477 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.i.service 2024-08-16T19:07:21.520 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.l.service 2024-08-16T19:07:21.563 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-16T19:07:21.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:21.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:21 smithi155 ceph-mon[37413]: Deploying daemon mds.l on smithi161 2024-08-16T19:07:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:21 smithi062 ceph-mon[32993]: Deploying daemon mds.l on smithi161 2024-08-16T19:07:21.829 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: Saving service mds.all spec with placement smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:21.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:21.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:21.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:21 smithi161 ceph-mon[37576]: Deploying daemon mds.l on smithi161 2024-08-16T19:07:22.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 -- 172.21.15.62:0/624237561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c107fd0 msgr2=0x7fd42c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:22.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 --2- 172.21.15.62:0/624237561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c107fd0 0x7fd42c10e970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd424009cd0 tx=0x7fd42402a8e0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 -- 172.21.15.62:0/624237561 shutdown_connections 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 --2- 172.21.15.62:0/624237561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c107fd0 0x7fd42c10e970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 --2- 172.21.15.62:0/624237561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c103410 0x7fd42c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 --2- 172.21.15.62:0/624237561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c102a60 0x7fd42c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 -- 172.21.15.62:0/624237561 >> 172.21.15.62:0/624237561 conn(0x7fd42c0fc500 msgr2=0x7fd42c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 -- 172.21.15.62:0/624237561 shutdown_connections 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.289+0000 7fd4309e6640 1 -- 172.21.15.62:0/624237561 wait complete. 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 Processor -- start 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 -- start start 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c103410 0x7fd42c100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c107fd0 0x7fd42c101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:22.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c111dd0 con 0x7fd42c102a60 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c111c50 con 0x7fd42c107fd0 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd4309e6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c111f50 con 0x7fd42c103410 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd42a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd42a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47442/0 (socket says 172.21.15.62:47442) 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd42a59c640 1 -- 172.21.15.62:0/3398988855 learned_addr learned my addr 172.21.15.62:0/3398988855 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.290+0000 7fd429d9b640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c103410 0x7fd42c100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 -- 172.21.15.62:0/3398988855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c103410 msgr2=0x7fd42c100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c103410 0x7fd42c100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 -- 172.21.15.62:0/3398988855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c107fd0 msgr2=0x7fd42c101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42ad9d640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c107fd0 0x7fd42c101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c107fd0 0x7fd42c101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 -- 172.21.15.62:0/3398988855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c1a6760 con 0x7fd42c102a60 2024-08-16T19:07:22.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd42a59c640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9f0 tx=0x7fd41c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00cd80 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c1a6a50 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41c005500 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c1a6f90 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.291+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c010690 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.292+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c005290 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.293+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd41c0046c0 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.294+0000 7fd41b7fe640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 0x7fd4140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.294+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd41c0cf1f0 con 0x7fd42c102a60 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.294+0000 7fd429d9b640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 0x7fd4140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.295+0000 7fd429d9b640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 0x7fd4140a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd42c102150 tx=0x7fd42000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:22.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.297+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd41c053df0 con 0x7fd42c102a60 2024-08-16T19:07:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.511+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd40c005580 con 0x7fd42c102a60 2024-08-16T19:07:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.512+0000 7fd41b7fe640 1 -- 172.21.15.62:0/3398988855 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fd41c09c5f0 con 0x7fd42c102a60 2024-08-16T19:07:22.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":1,"btime":"2024-08-16T19:01:02:785450+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-16T19:07:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 1 2024-08-16T19:07:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.513+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 msgr2=0x7fd4140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.513+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 0x7fd4140a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd42c102150 tx=0x7fd42000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 msgr2=0x7fd42c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00e9f0 tx=0x7fd41c00edf0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 shutdown_connections 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4140a2b80 0x7fd4140a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd42c107fd0 0x7fd42c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd42c103410 0x7fd42c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 --2- 172.21.15.62:0/3398988855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd42c102a60 0x7fd42c1006c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 >> 172.21.15.62:0/3398988855 conn(0x7fd42c0fc500 msgr2=0x7fd42c105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 shutdown_connections 2024-08-16T19:07:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:22.514+0000 7fd4309e6640 1 -- 172.21.15.62:0/3398988855 wait complete. 2024-08-16T19:07:22.888 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:22 smithi161 systemd[1]: Starting Ceph mds.l for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:22 smithi161 podman[66747]: 2024-08-16 19:07:22.823583355 +0000 UTC m=+0.017377497 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:22 smithi161 podman[66747]: 2024-08-16 19:07:22.933377756 +0000 UTC m=+0.127171879 container create 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 podman[66747]: 2024-08-16 19:07:23.14423758 +0000 UTC m=+0.338031704 container init 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 podman[66747]: 2024-08-16 19:07:23.146802677 +0000 UTC m=+0.340596801 container start 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mds[66766]: -- 172.21.15.161:0/3616077976 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5afeb3860 con 0x55c5afe91000 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: starting mds.l at 2024-08-16T19:07:23.228 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 bash[66747]: 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:07:23 smithi161 systemd[1]: Started Ceph mds.l for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3398988855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:boot 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: fsmap 1 up:standby 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:23.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:23.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:23 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3398988855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:boot 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: fsmap 1 up:standby 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:07:23.722 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.723 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.723 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.723 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:23.723 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:23.723 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:23 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3398988855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:boot 2024-08-16T19:07:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: fsmap 1 up:standby 2024-08-16T19:07:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:23.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:23 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:24.545 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:24 smithi161 ceph-mon[37576]: Deploying daemon mds.k on smithi155 2024-08-16T19:07:24.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:24 smithi062 ceph-mon[32993]: Deploying daemon mds.k on smithi155 2024-08-16T19:07:24.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:24 smithi155 ceph-mon[37413]: Deploying daemon mds.k on smithi155 2024-08-16T19:07:25.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:25.466 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 systemd[1]: Starting Ceph mds.k for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:25.483 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:25.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:25 smithi062 ceph-mon[32993]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:25.805 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:25 smithi155 ceph-mon[37413]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:25.805 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 podman[66771]: 2024-08-16 19:07:25.454225072 +0000 UTC m=+0.017214266 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:25.805 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 podman[66771]: 2024-08-16 19:07:25.552761025 +0000 UTC m=+0.115750211 container create 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux ) 2024-08-16T19:07:25.805 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 podman[66771]: 2024-08-16 19:07:25.805331198 +0000 UTC m=+0.368320397 container init 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:25.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:25 smithi161 ceph-mon[37576]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- 172.21.15.62:0/2524267876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54108010 msgr2=0x7fdc5410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 --2- 172.21.15.62:0/2524267876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54108010 0x7fdc5410e980 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009cd0 tx=0x7fdc4c02a910 comp rx=0 tx=0).stop 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- 172.21.15.62:0/2524267876 shutdown_connections 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 --2- 172.21.15.62:0/2524267876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54108010 0x7fdc5410e980 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 --2- 172.21.15.62:0/2524267876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 0x7fdc541078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 --2- 172.21.15.62:0/2524267876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54102ae0 0x7fdc54102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- 172.21.15.62:0/2524267876 >> 172.21.15.62:0/2524267876 conn(0x7fdc540fc500 msgr2=0x7fdc540fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- 172.21.15.62:0/2524267876 shutdown_connections 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- 172.21.15.62:0/2524267876 wait complete. 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 Processor -- start 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.060+0000 7fdc59563640 1 -- start start 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 0x7fdc541987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 0x7fdc5419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111e20 con 0x7fdc54102ae0 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111ca0 con 0x7fdc54108010 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc59563640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54111fa0 con 0x7fdc54103490 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47466/0 (socket says 172.21.15.62:47466) 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 -- 172.21.15.62:0/944664409 learned_addr learned my addr 172.21.15.62:0/944664409 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc537fe640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 0x7fdc5419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc527fc640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 0x7fdc541987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 -- 172.21.15.62:0/944664409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 msgr2=0x7fdc541987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 0x7fdc541987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 -- 172.21.15.62:0/944664409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 msgr2=0x7fdc5419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 0x7fdc5419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 -- 172.21.15.62:0/944664409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5419d260 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc537fe640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 0x7fdc5419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc52ffd640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c890 tx=0x7fdc4400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.061+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44004220 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.062+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5419d550 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.062+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc440043c0 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.062+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44004a00 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.062+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541a4e30 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.063+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc54102ec0 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.064+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc440079b0 con 0x7fdc54102ae0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.065+0000 7fdc3bfff640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.065+0000 7fdc527fc640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.065+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fdc440ce870 con 0x7fdc54102ae0 2024-08-16T19:07:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.065+0000 7fdc527fc640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc540feaf0 tx=0x7fdc48009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.069+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc44014030 con 0x7fdc54102ae0 2024-08-16T19:07:26.079 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 podman[66771]: 2024-08-16 19:07:25.808078664 +0000 UTC m=+0.371067856 container start 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, ceph=True) 2024-08-16T19:07:26.079 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 ceph-mds[66790]: -- 172.21.15.155:0/564173175 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d602bdd860 con 0x55d602bbb400 2024-08-16T19:07:26.079 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: starting mds.k at 2024-08-16T19:07:26.079 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 bash[66771]: 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 2024-08-16T19:07:26.079 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:07:25 smithi155 systemd[1]: Started Ceph mds.k for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:26.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.285+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc54109ff0 con 0x7fdc54102ae0 2024-08-16T19:07:26.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.286+0000 7fdc3bfff640 1 -- 172.21.15.62:0/944664409 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fdc440bb030 con 0x7fdc54102ae0 2024-08-16T19:07:26.286 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:26.286 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"btime":"2024-08-16T19:07:23:174532+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-16T19:07:26.287 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2024-08-16T19:07:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 msgr2=0x7fdc3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc540feaf0 tx=0x7fdc48009110 comp rx=0 tx=0).stop 2024-08-16T19:07:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 msgr2=0x7fdc541982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:26.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c890 tx=0x7fdc4400cc90 comp rx=0 tx=0).stop 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 shutdown_connections 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdc3c0a2b80 0x7fdc3c0a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc54108010 0x7fdc5419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc54103490 0x7fdc541987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 --2- 172.21.15.62:0/944664409 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc54102ae0 0x7fdc541982b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 >> 172.21.15.62:0/944664409 conn(0x7fdc540fc500 msgr2=0x7fdc541060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 shutdown_connections 2024-08-16T19:07:26.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:26.288+0000 7fdc59563640 1 -- 172.21.15.62:0/944664409 wait complete. 2024-08-16T19:07:27.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: Deploying daemon mds.j on smithi062 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/944664409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:boot 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: fsmap 2 up:standby 2024-08-16T19:07:27.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:26 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: Deploying daemon mds.j on smithi062 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/944664409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:boot 2024-08-16T19:07:27.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: fsmap 2 up:standby 2024-08-16T19:07:27.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:26 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: Deploying daemon mds.j on smithi062 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/944664409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:27.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:boot 2024-08-16T19:07:27.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: fsmap 2 up:standby 2024-08-16T19:07:27.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:26 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:07:28.746 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:28 smithi062 systemd[1]: Starting Ceph mds.j for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:28 smithi062 ceph-mon[32993]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:28 smithi062 podman[72056]: 2024-08-16 19:07:28.71223879 +0000 UTC m=+0.017823648 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:28 smithi062 podman[72056]: 2024-08-16 19:07:28.818233378 +0000 UTC m=+0.123818228 container create 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 podman[72056]: 2024-08-16 19:07:29.037646337 +0000 UTC m=+0.343231207 container init 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 podman[72056]: 2024-08-16 19:07:29.040723207 +0000 UTC m=+0.346308065 container start 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:07:29.065 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 ceph-mds[72076]: -- 172.21.15.62:0/746876428 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b04d31b860 con 0x55b04d2f8800 2024-08-16T19:07:29.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:28 smithi155 ceph-mon[37413]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:28 smithi161 ceph-mon[37576]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:29.496 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j[72072]: starting mds.j at 2024-08-16T19:07:29.496 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 bash[72056]: 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 2024-08-16T19:07:29.496 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:07:29 smithi062 systemd[1]: Started Ceph mds.j for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:29.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:30.218 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: Deploying daemon mds.d on smithi062 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6834/1364784278,v1:172.21.15.62:6835/1364784278] up:boot 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: fsmap 3 up:standby 2024-08-16T19:07:30.464 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:30 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: Deploying daemon mds.d on smithi062 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6834/1364784278,v1:172.21.15.62:6835/1364784278] up:boot 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: fsmap 3 up:standby 2024-08-16T19:07:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:30 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: Deploying daemon mds.d on smithi062 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6834/1364784278,v1:172.21.15.62:6835/1364784278] up:boot 2024-08-16T19:07:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: fsmap 3 up:standby 2024-08-16T19:07:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:30 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:07:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 -- 172.21.15.62:0/2322010330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 msgr2=0x7f444c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 --2- 172.21.15.62:0/2322010330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c1078c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4440009930 tx=0x7f444002a420 comp rx=0 tx=0).stop 2024-08-16T19:07:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 -- 172.21.15.62:0/2322010330 shutdown_connections 2024-08-16T19:07:31.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 --2- 172.21.15.62:0/2322010330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c107fd0 0x7f444c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 --2- 172.21.15.62:0/2322010330 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c1078c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 --2- 172.21.15.62:0/2322010330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c102a60 0x7f444c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 -- 172.21.15.62:0/2322010330 >> 172.21.15.62:0/2322010330 conn(0x7f444c0fc500 msgr2=0x7f444c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 -- 172.21.15.62:0/2322010330 shutdown_connections 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.111+0000 7f4451e33640 1 -- 172.21.15.62:0/2322010330 wait complete. 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.112+0000 7f4451e33640 1 Processor -- start 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.112+0000 7f4451e33640 1 -- start start 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c102a60 0x7f444c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c107fd0 0x7f444c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f444c111e40 con 0x7f444c103410 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f444c111cc0 con 0x7f444c102a60 2024-08-16T19:07:31.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f4451e33640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f444c111fc0 con 0x7f444c107fd0 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39440/0 (socket says 172.21.15.62:39440) 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 -- 172.21.15.62:0/441878257 learned_addr learned my addr 172.21.15.62:0/441878257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444b7fe640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c102a60 0x7f444c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444bfff640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c107fd0 0x7f444c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 -- 172.21.15.62:0/441878257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c107fd0 msgr2=0x7f444c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c107fd0 0x7f444c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 -- 172.21.15.62:0/441878257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c102a60 msgr2=0x7f444c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c102a60 0x7f444c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.113+0000 7f444affd640 1 -- 172.21.15.62:0/441878257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c19d290 con 0x7f444c103410 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f444affd640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f444002a940 tx=0x7f4440003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444003f070 con 0x7f444c103410 2024-08-16T19:07:31.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4440031c90 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c19d520 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440035030 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.114+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c1a4e60 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.115+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c06a580 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.116+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4440042070 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.117+0000 7f4448ff9640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 0x7f44340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.117+0000 7f444b7fe640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 0x7f44340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.117+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f44400ef3b0 con 0x7f444c103410 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.118+0000 7f444b7fe640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 0x7f44340a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f443c009990 tx=0x7f443c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:31.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.122+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f44400bc7b0 con 0x7f444c103410 2024-08-16T19:07:31.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.349+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f444c102e40 con 0x7f444c103410 2024-08-16T19:07:31.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.350+0000 7f4448ff9640 1 -- 172.21.15.62:0/441878257 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f44400bc0b0 con 0x7f444c103410 2024-08-16T19:07:31.351 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:31.351 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"btime":"2024-08-16T19:07:29:957414+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-16T19:07:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 4 2024-08-16T19:07:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.352+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 msgr2=0x7f44340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.352+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 0x7f44340a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f443c009990 tx=0x7f443c009110 comp rx=0 tx=0).stop 2024-08-16T19:07:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.352+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 msgr2=0x7f444c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.352+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f444002a940 tx=0x7f4440003c00 comp rx=0 tx=0).stop 2024-08-16T19:07:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 shutdown_connections 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f444c107fd0 0x7f444c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f444c103410 0x7f444c198820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f44340a2b80 0x7f44340a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 --2- 172.21.15.62:0/441878257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c102a60 0x7f444c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 >> 172.21.15.62:0/441878257 conn(0x7f444c0fc500 msgr2=0x7f444c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 shutdown_connections 2024-08-16T19:07:31.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:31.353+0000 7f4451e33640 1 -- 172.21.15.62:0/441878257 wait complete. 2024-08-16T19:07:31.364 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:31 smithi062 ceph-mon[32993]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:31 smithi155 ceph-mon[37413]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:31 smithi161 ceph-mon[37576]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:32.206 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 podman[72469]: 2024-08-16 19:07:32.102239034 +0000 UTC m=+0.126581372 container create 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux ) 2024-08-16T19:07:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:32 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/441878257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 podman[72469]: 2024-08-16 19:07:32.31320324 +0000 UTC m=+0.337545537 container init 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 podman[72469]: 2024-08-16 19:07:32.316259599 +0000 UTC m=+0.340601904 container start 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 ceph-mds[72489]: -- 172.21.15.62:0/1964320424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b619f4d860 con 0x55b619f2b400 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d[72485]: starting mds.d at 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 bash[72469]: 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b 2024-08-16T19:07:32.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:07:32 smithi062 systemd[1]: Started Ceph mds.d for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:32.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:32 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/441878257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:32 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/441878257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:33.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:33.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: Deploying daemon mds.e on smithi155 2024-08-16T19:07:33.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6836/3300090633,v1:172.21.15.62:6837/3300090633] up:boot 2024-08-16T19:07:33.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: fsmap 4 up:standby 2024-08-16T19:07:33.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:33 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:07:33.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: Deploying daemon mds.e on smithi155 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6836/3300090633,v1:172.21.15.62:6837/3300090633] up:boot 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: fsmap 4 up:standby 2024-08-16T19:07:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:33 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: Deploying daemon mds.e on smithi155 2024-08-16T19:07:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6836/3300090633,v1:172.21.15.62:6837/3300090633] up:boot 2024-08-16T19:07:33.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: fsmap 4 up:standby 2024-08-16T19:07:33.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:33 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:07:33.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:34.210 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:34.579 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 systemd[1]: Starting Ceph mds.e for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- 172.21.15.62:0/3295889365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f810e1b0 msgr2=0x7f13f81105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 --2- 172.21.15.62:0/3295889365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f810e1b0 0x7f13f81105a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009ca0 tx=0x7f13f002a8c0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- 172.21.15.62:0/3295889365 shutdown_connections 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 --2- 172.21.15.62:0/3295889365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f810e1b0 0x7f13f81105a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 --2- 172.21.15.62:0/3295889365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 0x7f13f810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 --2- 172.21.15.62:0/3295889365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f8100600 0x7f13f81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- 172.21.15.62:0/3295889365 >> 172.21.15.62:0/3295889365 conn(0x7f13f80fc4d0 msgr2=0x7f13f80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:34.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- 172.21.15.62:0/3295889365 shutdown_connections 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- 172.21.15.62:0/3295889365 wait complete. 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 Processor -- start 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.667+0000 7f13ff83f640 1 -- start start 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 0x7f13f819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 0x7f13f819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8113eb0 con 0x7f13f8100600 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8113d30 con 0x7f13f8100fb0 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8114030 con 0x7f13f810e1b0 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39482/0 (socket says 172.21.15.62:39482) 2024-08-16T19:07:34.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 -- 172.21.15.62:0/1010011632 learned_addr learned my addr 172.21.15.62:0/1010011632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fcdda640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 0x7f13f819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fdddc640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 0x7f13f819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 -- 172.21.15.62:0/1010011632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 msgr2=0x7f13f819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 0x7f13f819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 -- 172.21.15.62:0/1010011632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 msgr2=0x7f13f819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 0x7f13f819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 -- 172.21.15.62:0/1010011632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f819f380 con 0x7f13f8100600 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fdddc640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 0x7f13f819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13fd5db640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d950 tx=0x7f13f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f13f8100600 2024-08-16T19:07:34.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f4009c30 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f819f670 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.668+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f400f6b0 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.669+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f81a6f50 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.669+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f81009e0 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.671+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f13f40109d0 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.671+0000 7f13ee7fc640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.672+0000 7f13fcdda640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.672+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f13f40ce730 con 0x7f13f8100600 2024-08-16T19:07:34.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.672+0000 7f13fcdda640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f13f810eab0 tx=0x7f13e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:34.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.676+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f13f4009de0 con 0x7f13f8100600 2024-08-16T19:07:34.885 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 podman[67090]: 2024-08-16 19:07:34.521799846 +0000 UTC m=+0.017035885 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:34.885 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 podman[67090]: 2024-08-16 19:07:34.632428071 +0000 UTC m=+0.127664139 container create fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-16T19:07:34.885 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 podman[67090]: 2024-08-16 19:07:34.88549314 +0000 UTC m=+0.380729166 container init fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-16T19:07:34.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.887+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13f819b730 con 0x7f13f8100600 2024-08-16T19:07:34.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.888+0000 7f13ee7fc640 1 -- 172.21.15.62:0/1010011632 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f13f409bb30 con 0x7f13f8100600 2024-08-16T19:07:34.889 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:34.889 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"btime":"2024-08-16T19:07:33:206717+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-16T19:07:34.890 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 5 2024-08-16T19:07:34.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 msgr2=0x7f13e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:34.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f13f810eab0 tx=0x7f13e800b390 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 msgr2=0x7f13f819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13f400d950 tx=0x7f13f400de10 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 shutdown_connections 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13f810e1b0 0x7f13f819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f8100fb0 0x7f13f819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 --2- 172.21.15.62:0/1010011632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f8100600 0x7f13f819a3d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 >> 172.21.15.62:0/1010011632 conn(0x7f13f80fc4d0 msgr2=0x7f13f80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 shutdown_connections 2024-08-16T19:07:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:34.890+0000 7f13ff83f640 1 -- 172.21.15.62:0/1010011632 wait complete. 2024-08-16T19:07:35.329 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 podman[67090]: 2024-08-16 19:07:34.888024487 +0000 UTC m=+0.383260514 container start fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-16T19:07:35.329 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 ceph-mds[67110]: -- 172.21.15.155:0/3585288956 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3a99a5c20 con 0x55b3a9983400 2024-08-16T19:07:35.329 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: starting mds.e at 2024-08-16T19:07:35.329 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 bash[67090]: fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 2024-08-16T19:07:35.329 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:07:34 smithi155 systemd[1]: Started Ceph mds.e for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1010011632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:35.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:35.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:35 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1010011632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:35.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:35 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1010011632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:35.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:35.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:35 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:36 smithi062 ceph-mon[32993]: Deploying daemon mds.f on smithi161 2024-08-16T19:07:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:36 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6834/3063588307,v1:172.21.15.155:6835/3063588307] up:boot 2024-08-16T19:07:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:36 smithi062 ceph-mon[32993]: fsmap 5 up:standby 2024-08-16T19:07:36.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:36 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:07:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:36 smithi155 ceph-mon[37413]: Deploying daemon mds.f on smithi161 2024-08-16T19:07:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:36 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6834/3063588307,v1:172.21.15.155:6835/3063588307] up:boot 2024-08-16T19:07:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:36 smithi155 ceph-mon[37413]: fsmap 5 up:standby 2024-08-16T19:07:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:36 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:07:36.839 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:36 smithi161 ceph-mon[37576]: Deploying daemon mds.f on smithi161 2024-08-16T19:07:36.839 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:36 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6834/3063588307,v1:172.21.15.155:6835/3063588307] up:boot 2024-08-16T19:07:36.839 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:36 smithi161 ceph-mon[37576]: fsmap 5 up:standby 2024-08-16T19:07:36.839 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:36 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:07:37.138 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 systemd[1]: Starting Ceph mds.f for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:37.498 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:37 smithi161 ceph-mon[37576]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:37.498 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 podman[67065]: 2024-08-16 19:07:37.228871016 +0000 UTC m=+0.114911155 container create 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:37.499 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 podman[67065]: 2024-08-16 19:07:37.131578629 +0000 UTC m=+0.017618778 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:37.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:37 smithi062 ceph-mon[32993]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:37 smithi155 ceph-mon[37413]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 podman[67065]: 2024-08-16 19:07:37.498575488 +0000 UTC m=+0.384615627 container init 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d863d21, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 podman[67065]: 2024-08-16 19:07:37.501257701 +0000 UTC m=+0.387297841 container start 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux ) 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 ceph-mds[67084]: -- 172.21.15.161:0/2488307739 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a413d03860 con 0x55a413ce0800 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: starting mds.f at 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 bash[67065]: 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 2024-08-16T19:07:37.888 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:07:37 smithi161 systemd[1]: Started Ceph mds.f for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/4110801246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 msgr2=0x7fe3b01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/4110801246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b01026e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009900 tx=0x7fe3a402a480 comp rx=0 tx=0).stop 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/4110801246 shutdown_connections 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/4110801246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0102c20 0x7fe3b010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/4110801246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b01026e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/4110801246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0100b50 0x7fe3b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/4110801246 >> 172.21.15.62:0/4110801246 conn(0x7fe3b00fc500 msgr2=0x7fe3b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/4110801246 shutdown_connections 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.247+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/4110801246 wait complete. 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 Processor -- start 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 -- start start 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0100b50 0x7fe3b0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0102c20 0x7fe3b019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0111dd0 con 0x7fe3b01022a0 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0111c50 con 0x7fe3b0102c20 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b6b9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0111f50 con 0x7fe3b0100b50 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:38.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39502/0 (socket says 172.21.15.62:39502) 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3affff640 1 -- 172.21.15.62:0/1972960818 learned_addr learned my addr 172.21.15.62:0/1972960818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b493a640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0100b50 0x7fe3b0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.248+0000 7fe3b513b640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0102c20 0x7fe3b019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 -- 172.21.15.62:0/1972960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0100b50 msgr2=0x7fe3b0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0100b50 0x7fe3b0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 -- 172.21.15.62:0/1972960818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0102c20 msgr2=0x7fe3b019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0102c20 0x7fe3b019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 -- 172.21.15.62:0/1972960818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b019d1c0 con 0x7fe3b01022a0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3affff640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a403f070 con 0x7fe3b01022a0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a4031c30 con 0x7fe3b01022a0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b019d450 con 0x7fe3b01022a0 2024-08-16T19:07:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4035030 con 0x7fe3b01022a0 2024-08-16T19:07:38.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.249+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01a4d90 con 0x7fe3b01022a0 2024-08-16T19:07:38.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.250+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b0100f30 con 0x7fe3b01022a0 2024-08-16T19:07:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.252+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe3a4042070 con 0x7fe3b01022a0 2024-08-16T19:07:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.252+0000 7fe3adffb640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 0x7fe3980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.252+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fe3a40ef730 con 0x7fe3b01022a0 2024-08-16T19:07:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.252+0000 7fe3b493a640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 0x7fe3980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.253+0000 7fe3b493a640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 0x7fe3980a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000abe0 tx=0x7fe3a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:38.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.256+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe3a40bcb30 con 0x7fe3b01022a0 2024-08-16T19:07:38.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.468+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3b0199570 con 0x7fe3b01022a0 2024-08-16T19:07:38.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.468+0000 7fe3adffb640 1 -- 172.21.15.62:0/1972960818 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7fe3b0199570 con 0x7fe3b01022a0 2024-08-16T19:07:38.470 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:38.470 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":6,"btime":"2024-08-16T19:07:35:459916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-16T19:07:38.470 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 6 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 msgr2=0x7fe3980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 0x7fe3980a50c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000abe0 tx=0x7fe3a000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 msgr2=0x7fe3b0198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006010 tx=0x7fe3a4003c10 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 shutdown_connections 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3980a2c00 0x7fe3980a50c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b0102c20 0x7fe3b019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3b01022a0 0x7fe3b0198750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 --2- 172.21.15.62:0/1972960818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3b0100b50 0x7fe3b0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 >> 172.21.15.62:0/1972960818 conn(0x7fe3b00fc500 msgr2=0x7fe3b01065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:38.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 shutdown_connections 2024-08-16T19:07:38.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:38.471+0000 7fe3b6b9e640 1 -- 172.21.15.62:0/1972960818 wait complete. 2024-08-16T19:07:38.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:38.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: Deploying daemon mds.c on smithi161 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1972960818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6834/3346321531,v1:172.21.15.161:6835/3346321531] up:boot 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: fsmap 6 up:standby 2024-08-16T19:07:38.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:38 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:38.917 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: Deploying daemon mds.c on smithi161 2024-08-16T19:07:38.918 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1972960818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:38.918 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6834/3346321531,v1:172.21.15.161:6835/3346321531] up:boot 2024-08-16T19:07:38.918 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: fsmap 6 up:standby 2024-08-16T19:07:38.918 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:38 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: Deploying daemon mds.c on smithi161 2024-08-16T19:07:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1972960818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:39.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6834/3346321531,v1:172.21.15.161:6835/3346321531] up:boot 2024-08-16T19:07:39.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: fsmap 6 up:standby 2024-08-16T19:07:39.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:38 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:07:39.888 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:39 smithi161 systemd[1]: Starting Ceph mds.c for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:40.265 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:39 smithi161 podman[67365]: 2024-08-16 19:07:39.868362648 +0000 UTC m=+0.017360941 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:40.265 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 podman[67365]: 2024-08-16 19:07:40.003500215 +0000 UTC m=+0.152498537 container create 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True) 2024-08-16T19:07:40.265 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 podman[67365]: 2024-08-16 19:07:40.265024729 +0000 UTC m=+0.414023009 container init 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:40.638 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 podman[67365]: 2024-08-16 19:07:40.26765893 +0000 UTC m=+0.416657211 container start 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:40.638 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 ceph-mds[67384]: -- 172.21.15.161:0/3822555786 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a29713b860 con 0x55a297119000 2024-08-16T19:07:40.638 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: starting mds.c at 2024-08-16T19:07:40.638 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 bash[67365]: 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef 2024-08-16T19:07:40.638 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:07:40 smithi161 systemd[1]: Started Ceph mds.c for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:41.096 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:41.374 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:41.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:41.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:41.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: Deploying daemon mds.h on smithi155 2024-08-16T19:07:41.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:boot 2024-08-16T19:07:41.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: fsmap 7 up:standby 2024-08-16T19:07:41.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:41 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:41.735 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: Deploying daemon mds.h on smithi155 2024-08-16T19:07:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:boot 2024-08-16T19:07:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: fsmap 7 up:standby 2024-08-16T19:07:41.736 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:41 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 -- 172.21.15.62:0/2098188244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f0102260 msgr2=0x7fb1f010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2098188244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f0102260 0x7fb1f010e790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009cd0 tx=0x7fb1e802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 -- 172.21.15.62:0/2098188244 shutdown_connections 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2098188244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f0102260 0x7fb1f010e790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2098188244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f01018c0 0x7fb1f0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2098188244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01078c0 0x7fb1f0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 -- 172.21.15.62:0/2098188244 >> 172.21.15.62:0/2098188244 conn(0x7fb1f00fd5e0 msgr2=0x7fb1f00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 -- 172.21.15.62:0/2098188244 shutdown_connections 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.816+0000 7fb1f7701640 1 -- 172.21.15.62:0/2098188244 wait complete. 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 Processor -- start 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 -- start start 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01018c0 0x7fb1f0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 0x7fb1f01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0111cc0 con 0x7fb1f01078c0 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0111b40 con 0x7fb1f0102260 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f7701640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0111e40 con 0x7fb1f01018c0 2024-08-16T19:07:41.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.817+0000 7fb1f5c9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48058/0 (socket says 172.21.15.62:48058) 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 -- 172.21.15.62:0/2497003388 learned_addr learned my addr 172.21.15.62:0/2497003388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f549d640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01018c0 0x7fb1f0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f4c9c640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 0x7fb1f01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 -- 172.21.15.62:0/2497003388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01018c0 msgr2=0x7fb1f0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01018c0 0x7fb1f0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 -- 172.21.15.62:0/2497003388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 msgr2=0x7fb1f01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 0x7fb1f01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 -- 172.21.15.62:0/2497003388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f019d030 con 0x7fb1f01078c0 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f4c9c640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 0x7fb1f01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:07:41.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f5c9e640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005cc0 tx=0x7fb1e8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8004040 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e80046c0 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f019d2c0 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8046480 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.818+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a4c00 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.819+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0101d20 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.820+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb1e80305a0 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.821+0000 7fb1e27fc640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 0x7fb1dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.821+0000 7fb1f549d640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 0x7fb1dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.821+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb1e802e030 con 0x7fb1f01078c0 2024-08-16T19:07:41.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.822+0000 7fb1f549d640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 0x7fb1dc0a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005fd0 tx=0x7fb1e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:41.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:41.824+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb1e804b060 con 0x7fb1f01078c0 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:41.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:41.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: Deploying daemon mds.h on smithi155 2024-08-16T19:07:41.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:boot 2024-08-16T19:07:41.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: fsmap 7 up:standby 2024-08-16T19:07:41.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:41 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:07:42.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.035+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f01993e0 con 0x7fb1f01078c0 2024-08-16T19:07:42.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.036+0000 7fb1e27fc640 1 -- 172.21.15.62:0/2497003388 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7fb1f01993e0 con 0x7fb1f01078c0 2024-08-16T19:07:42.038 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:42.038 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":8,"btime":"2024-08-16T19:07:40:647236+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-16T19:07:42.038 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 8 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 msgr2=0x7fb1dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 0x7fb1dc0a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4005fd0 tx=0x7fb1e400b390 comp rx=0 tx=0).stop 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 msgr2=0x7fb1f019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005cc0 tx=0x7fb1e8003970 comp rx=0 tx=0).stop 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 shutdown_connections 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1f01078c0 0x7fb1f019c950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb1dc0a2c00 0x7fb1dc0a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1f0102260 0x7fb1f01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 --2- 172.21.15.62:0/2497003388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1f01018c0 0x7fb1f0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 >> 172.21.15.62:0/2497003388 conn(0x7fb1f00fd5e0 msgr2=0x7fb1f00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 shutdown_connections 2024-08-16T19:07:42.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:42.038+0000 7fb1f7701640 1 -- 172.21.15.62:0/2497003388 wait complete. 2024-08-16T19:07:42.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:42 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2497003388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:42.579 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 systemd[1]: Starting Ceph mds.h for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:42.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:42 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2497003388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:42.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:42 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2497003388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:42.890 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 podman[67393]: 2024-08-16 19:07:42.524509039 +0000 UTC m=+0.017211491 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:42.891 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 podman[67393]: 2024-08-16 19:07:42.646432059 +0000 UTC m=+0.139134492 container create cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:07:42.891 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 podman[67393]: 2024-08-16 19:07:42.891108523 +0000 UTC m=+0.383810956 container init cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True) 2024-08-16T19:07:43.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 podman[67393]: 2024-08-16 19:07:42.89357377 +0000 UTC m=+0.386276206 container start cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-16T19:07:43.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 ceph-mds[67412]: -- 172.21.15.155:0/287660118 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564994183860 con 0x564994160800 2024-08-16T19:07:43.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: starting mds.h at 2024-08-16T19:07:43.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 bash[67393]: cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 2024-08-16T19:07:43.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:07:42 smithi155 systemd[1]: Started Ceph mds.h for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:43.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:43 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:43.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:43.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:43 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:43 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:44.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:44 smithi161 ceph-mon[37576]: Deploying daemon mds.i on smithi161 2024-08-16T19:07:44.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:44 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6836/3410199866,v1:172.21.15.155:6837/3410199866] up:boot 2024-08-16T19:07:44.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:44 smithi161 ceph-mon[37576]: fsmap 8 up:standby 2024-08-16T19:07:44.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:44 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:07:44.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:44.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:44 smithi062 ceph-mon[32993]: Deploying daemon mds.i on smithi161 2024-08-16T19:07:44.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:44 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6836/3410199866,v1:172.21.15.155:6837/3410199866] up:boot 2024-08-16T19:07:44.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:44 smithi062 ceph-mon[32993]: fsmap 8 up:standby 2024-08-16T19:07:44.708 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:44 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:07:44.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:44 smithi155 ceph-mon[37413]: Deploying daemon mds.i on smithi161 2024-08-16T19:07:44.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:44 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6836/3410199866,v1:172.21.15.155:6837/3410199866] up:boot 2024-08-16T19:07:44.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:44 smithi155 ceph-mon[37413]: fsmap 8 up:standby 2024-08-16T19:07:44.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:44 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:07:44.955 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:45.055 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 systemd[1]: Starting Ceph mds.i for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:45.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 podman[67678]: 2024-08-16 19:07:45.175881165 +0000 UTC m=+0.017618552 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:45.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 podman[67678]: 2024-08-16 19:07:45.29314974 +0000 UTC m=+0.134887127 container create e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-16T19:07:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 -- 172.21.15.62:0/1149812558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8102a60 msgr2=0x7f00e8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:45.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 --2- 172.21.15.62:0/1149812558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8102a60 0x7f00e8102e40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009930 tx=0x7f00d802a420 comp rx=0 tx=0).stop 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 -- 172.21.15.62:0/1149812558 shutdown_connections 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 --2- 172.21.15.62:0/1149812558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8107fd0 0x7f00e810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 --2- 172.21.15.62:0/1149812558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 0x7f00e81078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 --2- 172.21.15.62:0/1149812558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8102a60 0x7f00e8102e40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 -- 172.21.15.62:0/1149812558 >> 172.21.15.62:0/1149812558 conn(0x7f00e80fc500 msgr2=0x7f00e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 -- 172.21.15.62:0/1149812558 shutdown_connections 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.410+0000 7f00ec909640 1 -- 172.21.15.62:0/1149812558 wait complete. 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 Processor -- start 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 -- start start 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 0x7f00e81006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 0x7f00e8100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:45.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111dd0 con 0x7f00e8107fd0 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111c50 con 0x7f00e8103410 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00ec909640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111f50 con 0x7f00e8102a60 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00e659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 0x7f00e81006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00e6d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48072/0 (socket says 172.21.15.62:48072) 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00e6d9d640 1 -- 172.21.15.62:0/3807511743 learned_addr learned my addr 172.21.15.62:0/3807511743 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.411+0000 7f00e659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 0x7f00e81006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:57972/0 (socket says 172.21.15.62:57972) 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e5d9b640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 0x7f00e8100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 -- 172.21.15.62:0/3807511743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 msgr2=0x7f00e81006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 0x7f00e81006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 -- 172.21.15.62:0/3807511743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 msgr2=0x7f00e8100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 0x7f00e8100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 -- 172.21.15.62:0/3807511743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81a6790 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00e6d9d640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eaa0 tx=0x7f00e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e000cdb0 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0005020 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a6a80 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e00116b0 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.412+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a6fc0 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.413+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e806a580 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.414+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00e0005560 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.415+0000 7f00d77fe640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.415+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f00e00cf0c0 con 0x7f00e8107fd0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.415+0000 7f00e659c640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:45.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.416+0000 7f00e659c640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 0x7f00d00a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.418+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f00e0016030 con 0x7f00e8107fd0 2024-08-16T19:07:45.608 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:45 smithi062 ceph-mon[32993]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.636+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00e8102e40 con 0x7f00e8107fd0 2024-08-16T19:07:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.637+0000 7f00d77fe640 1 -- 172.21.15.62:0/3807511743 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f00e009c540 con 0x7f00e8107fd0 2024-08-16T19:07:45.638 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:45.638 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":9,"btime":"2024-08-16T19:07:43:402299+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-16T19:07:45.638 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 9 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.639+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 msgr2=0x7f00d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.639+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 0x7f00d00a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d80058d0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.639+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 msgr2=0x7f00e8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.639+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00e000eaa0 tx=0x7f00e000ef60 comp rx=0 tx=0).stop 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 shutdown_connections 2024-08-16T19:07:45.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00e8107fd0 0x7f00e8101140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00e8103410 0x7f00e8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 --2- 172.21.15.62:0/3807511743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f00e8102a60 0x7f00e81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 >> 172.21.15.62:0/3807511743 conn(0x7f00e80fc500 msgr2=0x7f00e8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 shutdown_connections 2024-08-16T19:07:45.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:45.640+0000 7f00ec909640 1 -- 172.21.15.62:0/3807511743 wait complete. 2024-08-16T19:07:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:45 smithi155 ceph-mon[37413]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:45 smithi161 ceph-mon[37576]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 podman[67678]: 2024-08-16 19:07:45.537466017 +0000 UTC m=+0.379203403 container init e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 podman[67678]: 2024-08-16 19:07:45.540154854 +0000 UTC m=+0.381892241 container start e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 ceph-mds[67697]: -- 172.21.15.161:0/870103870 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a0df5b860 con 0x558a0df39400 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: starting mds.i at 2024-08-16T19:07:45.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 bash[67678]: e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 2024-08-16T19:07:45.889 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:07:45 smithi161 systemd[1]: Started Ceph mds.i for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3807511743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:46.677 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:46 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3807511743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:46 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:46.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3807511743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:46.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:46 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:47 smithi062 ceph-mon[32993]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:47 smithi062 ceph-mon[32993]: Deploying daemon mds.g on smithi062 2024-08-16T19:07:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:47 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:boot 2024-08-16T19:07:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:47 smithi062 ceph-mon[32993]: fsmap 9 up:standby 2024-08-16T19:07:47.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:47 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:07:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:47 smithi155 ceph-mon[37413]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:47 smithi155 ceph-mon[37413]: Deploying daemon mds.g on smithi062 2024-08-16T19:07:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:47 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:boot 2024-08-16T19:07:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:47 smithi155 ceph-mon[37413]: fsmap 9 up:standby 2024-08-16T19:07:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:47 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:07:47.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:47 smithi161 ceph-mon[37576]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:47 smithi161 ceph-mon[37576]: Deploying daemon mds.g on smithi062 2024-08-16T19:07:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:47 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:boot 2024-08-16T19:07:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:47 smithi161 ceph-mon[37576]: fsmap 9 up:standby 2024-08-16T19:07:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:47 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:07:48.239 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 systemd[1]: Starting Ceph mds.g for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:48.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:48.496 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 podman[73246]: 2024-08-16 19:07:48.141578284 +0000 UTC m=+0.017735689 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:48.496 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 podman[73246]: 2024-08-16 19:07:48.252079139 +0000 UTC m=+0.128236545 container create 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-16T19:07:48.702 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:48.826 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 podman[73246]: 2024-08-16 19:07:48.496766481 +0000 UTC m=+0.372923901 container init 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:07:48.826 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 podman[73246]: 2024-08-16 19:07:48.499975593 +0000 UTC m=+0.376133007 container start 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:07:48.827 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 ceph-mds[73284]: -- 172.21.15.62:0/492912902 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5617bd34f860 con 0x5617bd32c800 2024-08-16T19:07:48.827 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: starting mds.g at 2024-08-16T19:07:48.827 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 bash[73246]: 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d 2024-08-16T19:07:48.827 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:07:48 smithi062 systemd[1]: Started Ceph mds.g for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 -- 172.21.15.62:0/2399406462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 msgr2=0x7f1d6010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 --2- 172.21.15.62:0/2399406462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d6010e330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ca0 tx=0x7f1d5802a950 comp rx=0 tx=0).stop 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 -- 172.21.15.62:0/2399406462 shutdown_connections 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 --2- 172.21.15.62:0/2399406462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d6010e330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 --2- 172.21.15.62:0/2399406462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 0x7f1d6010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 --2- 172.21.15.62:0/2399406462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 0x7f1d60102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 -- 172.21.15.62:0/2399406462 >> 172.21.15.62:0/2399406462 conn(0x7f1d600fc4d0 msgr2=0x7f1d600fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 -- 172.21.15.62:0/2399406462 shutdown_connections 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.204+0000 7f1d666b7640 1 -- 172.21.15.62:0/2399406462 wait complete. 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 Processor -- start 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 -- start start 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 0x7f1d60100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 0x7f1d60100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111c70 con 0x7f1d6010beb0 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111af0 con 0x7f1d60102ad0 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d666b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111df0 con 0x7f1d601033f0 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d64c54640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d64c54640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48104/0 (socket says 172.21.15.62:48104) 2024-08-16T19:07:49.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d64c54640 1 -- 172.21.15.62:0/516043608 learned_addr learned my addr 172.21.15.62:0/516043608 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.205+0000 7f1d5ffff640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 0x7f1d60100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d5f7fe640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 0x7f1d60100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 -- 172.21.15.62:0/516043608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 msgr2=0x7f1d60100c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 0x7f1d60100c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 -- 172.21.15.62:0/516043608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 msgr2=0x7f1d60100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 0x7f1d60100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 -- 172.21.15.62:0/516043608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601a65c0 con 0x7f1d6010beb0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d64c54640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d5802ae70 tx=0x7f1d5802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58034040 con 0x7f1d6010beb0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a6850 con 0x7f1d6010beb0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d58030920 con 0x7f1d6010beb0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58045480 con 0x7f1d6010beb0 2024-08-16T19:07:49.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.206+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a6d30 con 0x7f1d6010beb0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.207+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d60102eb0 con 0x7f1d6010beb0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.208+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d58037020 con 0x7f1d6010beb0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.209+0000 7f1d5d7fa640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 0x7f1d480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.209+0000 7f1d5ffff640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 0x7f1d480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.209+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f1d580efd00 con 0x7f1d6010beb0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.209+0000 7f1d5ffff640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 0x7f1d480a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000bbe0 tx=0x7f1d5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:49.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.212+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d58080df0 con 0x7f1d6010beb0 2024-08-16T19:07:49.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.437+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6002d5e0 con 0x7f1d6010beb0 2024-08-16T19:07:49.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.440+0000 7f1d5d7fa640 1 -- 172.21.15.62:0/516043608 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f1d580bd060 con 0x7f1d6010beb0 2024-08-16T19:07:49.441 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:49.441 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":11,"btime":"2024-08-16T19:07:49:426343+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-16T19:07:49.441 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 11 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 msgr2=0x7f1d480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 0x7f1d480a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000bbe0 tx=0x7f1d5000b390 comp rx=0 tx=0).stop 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 msgr2=0x7f1d601011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d5802ae70 tx=0x7f1d5802f5b0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 shutdown_connections 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d6010beb0 0x7f1d601011b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d601033f0 0x7f1d60100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1d480a2b80 0x7f1d480a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 --2- 172.21.15.62:0/516043608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d60102ad0 0x7f1d60100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:49.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.442+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 >> 172.21.15.62:0/516043608 conn(0x7f1d600fc4d0 msgr2=0x7f1d600fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:49.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.443+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 shutdown_connections 2024-08-16T19:07:49.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:49.443+0000 7f1d666b7640 1 -- 172.21.15.62:0/516043608 wait complete. 2024-08-16T19:07:49.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:49.721 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:49 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:49.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:49.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:49 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:49.891 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:49.891 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.891 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.891 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:49.892 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:49.892 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:49.892 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:49 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:50.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:50 smithi062 ceph-mon[32993]: Deploying daemon mds.a on smithi062 2024-08-16T19:07:50.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:50 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6838/659126409,v1:172.21.15.62:6839/659126409] up:boot 2024-08-16T19:07:50.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:50 smithi062 ceph-mon[32993]: fsmap 10 up:standby 2024-08-16T19:07:50.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:50 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:07:50.506 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:50 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/516043608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:50 smithi155 ceph-mon[37413]: Deploying daemon mds.a on smithi062 2024-08-16T19:07:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:50 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6838/659126409,v1:172.21.15.62:6839/659126409] up:boot 2024-08-16T19:07:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:50 smithi155 ceph-mon[37413]: fsmap 10 up:standby 2024-08-16T19:07:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:50 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:07:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:50 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/516043608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:50 smithi161 ceph-mon[37576]: Deploying daemon mds.a on smithi062 2024-08-16T19:07:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:50 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6838/659126409,v1:172.21.15.62:6839/659126409] up:boot 2024-08-16T19:07:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:50 smithi161 ceph-mon[37576]: fsmap 10 up:standby 2024-08-16T19:07:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:50 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:07:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:50 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/516043608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:51.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:51 smithi062 ceph-mon[32993]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:51 smithi155 ceph-mon[37413]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:51 smithi161 ceph-mon[37576]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:51.906 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:07:51 smithi062 podman[73663]: 2024-08-16 19:07:51.88204641 +0000 UTC m=+0.336572019 container start cb9791b7d2cba6a33dba6c96933be62b874b6a74fb696c2274dc060e512fe81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-16T19:07:52.246 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:07:51 smithi062 ceph-mds[73683]: -- 172.21.15.62:0/2884547973 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9cd547860 con 0x55e9cd525000 2024-08-16T19:07:52.246 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:07:51 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: starting mds.a at 2024-08-16T19:07:52.247 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:07:51 smithi062 bash[73663]: cb9791b7d2cba6a33dba6c96933be62b874b6a74fb696c2274dc060e512fe81f 2024-08-16T19:07:52.247 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:07:51 smithi062 systemd[1]: Started Ceph mds.a for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:52.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:53.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:53.241 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: Deploying daemon mds.b on smithi155 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6840/3131957026,v1:172.21.15.62:6841/3131957026] up:boot 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: fsmap 11 up:standby 2024-08-16T19:07:53.242 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:53 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:07:53.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 -- 172.21.15.62:0/4066406311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 msgr2=0x7f5734101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:53.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 --2- 172.21.15.62:0/4066406311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f5734101d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5728009930 tx=0x7f572802a420 comp rx=0 tx=0).stop 2024-08-16T19:07:53.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 -- 172.21.15.62:0/4066406311 shutdown_connections 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 --2- 172.21.15.62:0/4066406311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5734102250 0x7f573410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 --2- 172.21.15.62:0/4066406311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f5734101d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 --2- 172.21.15.62:0/4066406311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57341078b0 0x7f5734107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 -- 172.21.15.62:0/4066406311 >> 172.21.15.62:0/4066406311 conn(0x7f57340fd5f0 msgr2=0x7f57340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 -- 172.21.15.62:0/4066406311 shutdown_connections 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.240+0000 7f573c2bf640 1 -- 172.21.15.62:0/4066406311 wait complete. 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.241+0000 7f573c2bf640 1 Processor -- start 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.241+0000 7f573c2bf640 1 -- start start 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.241+0000 7f573c2bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.241+0000 7f573c2bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734102250 0x7f573419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341078b0 0x7f573419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5734111cf0 con 0x7f57341018b0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5734111b70 con 0x7f57341078b0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5734111e70 con 0x7f5734102250 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55540/0 (socket says 172.21.15.62:55540) 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 -- 172.21.15.62:0/3242344331 learned_addr learned my addr 172.21.15.62:0/3242344331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573985a640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734102250 0x7f573419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a85c640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341078b0 0x7f573419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 -- 172.21.15.62:0/3242344331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734102250 msgr2=0x7f573419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734102250 0x7f573419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 -- 172.21.15.62:0/3242344331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341078b0 msgr2=0x7f573419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341078b0 0x7f573419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 -- 172.21.15.62:0/3242344331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573419f3c0 con 0x7f57341018b0 2024-08-16T19:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573a05b640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f573000d770 tx=0x7f573000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730017070 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5730005500 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573419f6b0 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730004e00 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.242+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a6f90 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.243+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734102ff0 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.245+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5730004fa0 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.245+0000 7f57277fe640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 0x7f57200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.246+0000 7f573985a640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 0x7f57200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.246+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f57300ce620 con 0x7f57341018b0 2024-08-16T19:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.246+0000 7f573985a640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 0x7f57200a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5728000c00 tx=0x7f5728005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.250+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f573009ba20 con 0x7f57341018b0 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: Deploying daemon mds.b on smithi155 2024-08-16T19:07:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6840/3131957026,v1:172.21.15.62:6841/3131957026] up:boot 2024-08-16T19:07:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: fsmap 11 up:standby 2024-08-16T19:07:53.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:53 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: Deploying daemon mds.b on smithi155 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6840/3131957026,v1:172.21.15.62:6841/3131957026] up:boot 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: fsmap 11 up:standby 2024-08-16T19:07:53.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:53 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:07:53.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.461+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57341a7160 con 0x7f57341018b0 2024-08-16T19:07:53.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.462+0000 7f57277fe640 1 -- 172.21.15.62:0/3242344331 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7f573009b320 con 0x7f57341018b0 2024-08-16T19:07:53.462 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:53.463 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":12,"btime":"2024-08-16T19:07:52:444988+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-16T19:07:53.463 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 12 2024-08-16T19:07:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 msgr2=0x7f57200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 0x7f57200a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5728000c00 tx=0x7f5728005a20 comp rx=0 tx=0).stop 2024-08-16T19:07:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 msgr2=0x7f573419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:53.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f573000d770 tx=0x7f573000dc30 comp rx=0 tx=0).stop 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 shutdown_connections 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f57200a2c00 0x7f57200a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341078b0 0x7f573419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734102250 0x7f573419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 --2- 172.21.15.62:0/3242344331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341018b0 0x7f573419a410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 >> 172.21.15.62:0/3242344331 conn(0x7f57340fd5f0 msgr2=0x7f573410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 shutdown_connections 2024-08-16T19:07:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:53.464+0000 7f573c2bf640 1 -- 172.21.15.62:0/3242344331 wait complete. 2024-08-16T19:07:54.239 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:54 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3242344331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:54.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:54 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3242344331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:54.255 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 systemd[1]: Starting Ceph mds.b for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:07:54.255 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 podman[67694]: 2024-08-16 19:07:54.156382973 +0000 UTC m=+0.017496073 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:07:54.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:54 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3242344331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:54.512 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 podman[67694]: 2024-08-16 19:07:54.284957694 +0000 UTC m=+0.146070803 container create 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-d863d21) 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 podman[67694]: 2024-08-16 19:07:54.512595719 +0000 UTC m=+0.373708826 container init 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 podman[67694]: 2024-08-16 19:07:54.515092395 +0000 UTC m=+0.376205496 container start 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 ceph-mds[67714]: -- 172.21.15.155:0/3325620869 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e336935860 con 0x55e336913400 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: starting mds.b at 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 bash[67694]: 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 2024-08-16T19:07:54.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:07:54 smithi155 systemd[1]: Started Ceph mds.b for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:55 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.377 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:55 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:55 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:56.273 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:07:56.298 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:56 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:boot 2024-08-16T19:07:56.298 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:56 smithi062 ceph-mon[32993]: fsmap 12 up:standby 2024-08-16T19:07:56.298 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:56 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:07:56.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:56 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:boot 2024-08-16T19:07:56.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:56 smithi155 ceph-mon[37413]: fsmap 12 up:standby 2024-08-16T19:07:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:56 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:07:56.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:56 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:boot 2024-08-16T19:07:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:56 smithi161 ceph-mon[37576]: fsmap 12 up:standby 2024-08-16T19:07:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:56 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:07:56.559 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 -- 172.21.15.62:0/4002807549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 msgr2=0x7f82081026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 --2- 172.21.15.62:0/4002807549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f82081026e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02a480 comp rx=0 tx=0).stop 2024-08-16T19:07:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 -- 172.21.15.62:0/4002807549 shutdown_connections 2024-08-16T19:07:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 --2- 172.21.15.62:0/4002807549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208102c20 0x7f820810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 --2- 172.21.15.62:0/4002807549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f82081026e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 --2- 172.21.15.62:0/4002807549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208100b50 0x7f8208100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 -- 172.21.15.62:0/4002807549 >> 172.21.15.62:0/4002807549 conn(0x7f82080fc500 msgr2=0x7f82080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 -- 172.21.15.62:0/4002807549 shutdown_connections 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.048+0000 7f820dd33640 1 -- 172.21.15.62:0/4002807549 wait complete. 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 Processor -- start 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 -- start start 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 0x7f8208199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f8208199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 0x7f82081a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8208110cb0 con 0x7f82081022a0 2024-08-16T19:07:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8208110b30 con 0x7f8208100b50 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f820dd33640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8208110e30 con 0x7f8208102c20 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f8208199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f8208199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55558/0 (socket says 172.21.15.62:55558) 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f8206ffd640 1 -- 172.21.15.62:0/506045095 learned_addr learned my addr 172.21.15.62:0/506045095 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f8207fff640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 0x7f82081a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.049+0000 7f82077fe640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 0x7f8208199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 -- 172.21.15.62:0/506045095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 msgr2=0x7f82081a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 0x7f82081a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 -- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 msgr2=0x7f8208199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f8208199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 -- 172.21.15.62:0/506045095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081a46c0 con 0x7f8208100b50 2024-08-16T19:07:57.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f8207fff640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 0x7f82081a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f82077fe640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 0x7f8208199100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81f800e9f0 tx=0x7f81f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.050+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f800ccc0 con 0x7f8208100b50 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.051+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82081a4890 con 0x7f8208100b50 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.051+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f8005020 con 0x7f8208100b50 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.051+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081a4da0 con 0x7f8208100b50 2024-08-16T19:07:57.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.051+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f80049e0 con 0x7f8208100b50 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.052+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f4005290 con 0x7f8208100b50 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.053+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f81f8004cc0 con 0x7f8208100b50 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.054+0000 7f8204ff9640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.054+0000 7f8206ffd640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.054+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f81f80ce9e0 con 0x7f8208100b50 2024-08-16T19:07:57.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.055+0000 7f8206ffd640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 0x7f81f00a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005770 tx=0x7f81fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:57.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.057+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f81f809bd40 con 0x7f8208100b50 2024-08-16T19:07:57.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.285+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81f4005580 con 0x7f8208100b50 2024-08-16T19:07:57.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.286+0000 7f8204ff9640 1 -- 172.21.15.62:0/506045095 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10536 (secure 0 0 0) 0x7f81f8097020 con 0x7f8208100b50 2024-08-16T19:07:57.287 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:57.287 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":13,"btime":"2024-08-16T19:07:55:034189+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-16T19:07:57.287 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 13 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 msgr2=0x7f81f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 0x7f81f00a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005770 tx=0x7f81fc005a20 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 msgr2=0x7f8208199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 0x7f8208199100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81f800e9f0 tx=0x7f81f800eeb0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 shutdown_connections 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8208102c20 0x7f82081a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82081022a0 0x7f8208199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 --2- 172.21.15.62:0/506045095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8208100b50 0x7f8208199100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:57.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 >> 172.21.15.62:0/506045095 conn(0x7f82080fc500 msgr2=0x7f82080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:57.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 shutdown_connections 2024-08-16T19:07:57.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:57.288+0000 7f820dd33640 1 -- 172.21.15.62:0/506045095 wait complete. 2024-08-16T19:07:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:57 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:57.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.555 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:57 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:57.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:57 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:07:58.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-16T19:07:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:58 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/506045095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:58.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:58 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/506045095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:58.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:58.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:58 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/506045095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:07:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:58.680 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 -- 172.21.15.62:0/422603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d41033d0 msgr2=0x7f41d410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 --2- 172.21.15.62:0/422603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d41033d0 0x7f41d410b8c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009900 tx=0x7f41bc02a480 comp rx=0 tx=0).stop 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 -- 172.21.15.62:0/422603413 shutdown_connections 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 --2- 172.21.15.62:0/422603413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 0x7f41d410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 --2- 172.21.15.62:0/422603413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d41033d0 0x7f41d410b8c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 --2- 172.21.15.62:0/422603413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d4102ab0 0x7f41d4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 -- 172.21.15.62:0/422603413 >> 172.21.15.62:0/422603413 conn(0x7f41d40fc4b0 msgr2=0x7f41d40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:59.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 -- 172.21.15.62:0/422603413 shutdown_connections 2024-08-16T19:07:59.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 -- 172.21.15.62:0/422603413 wait complete. 2024-08-16T19:07:59.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.175+0000 7f41d950e640 1 Processor -- start 2024-08-16T19:07:59.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 -- start start 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d41033d0 0x7f41d4100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 0x7f41d4101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4111c70 con 0x7f41d4102ab0 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4111af0 con 0x7f41d41033d0 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d950e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4111df0 con 0x7f41d410be90 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55566/0 (socket says 172.21.15.62:55566) 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 -- 172.21.15.62:0/992376215 learned_addr learned my addr 172.21.15.62:0/992376215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d37fe640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 0x7f41d4101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41ca7fc640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d41033d0 0x7f41d4100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 -- 172.21.15.62:0/992376215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 msgr2=0x7f41d4101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 0x7f41d4101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 -- 172.21.15.62:0/992376215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d41033d0 msgr2=0x7f41d4100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.176+0000 7f41d2ffd640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d41033d0 0x7f41d4100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d2ffd640 1 -- 172.21.15.62:0/992376215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d41a65b0 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d2ffd640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c920 tx=0x7f41c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4007c70 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4007e10 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d41a68a0 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4018890 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.177+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d41a6db0 con 0x7f41d4102ab0 2024-08-16T19:07:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.178+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d4102e90 con 0x7f41d4102ab0 2024-08-16T19:07:59.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.180+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41c4018a50 con 0x7f41d4102ab0 2024-08-16T19:07:59.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.181+0000 7f41d0ff9640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 0x7f41c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:07:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.181+0000 7f41ca7fc640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 0x7f41c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:07:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.181+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f41c40ceed0 con 0x7f41d4102ab0 2024-08-16T19:07:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.182+0000 7f41ca7fc640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 0x7f41c00a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009230 tx=0x7f41bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:07:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.184+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f41c4014030 con 0x7f41d4102ab0 2024-08-16T19:07:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.379+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f41d402d5e0 con 0x7f41d4102ab0 2024-08-16T19:07:59.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:59 smithi062 ceph-mon[32993]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:59.380 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.381 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.381 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.381 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:07:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.380+0000 7f41d0ff9640 1 -- 172.21.15.62:0/992376215 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f41c409c360 con 0x7f41d4102ab0 2024-08-16T19:07:59.381 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:07:59.381 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-08-16T19:07:59.381 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:07:59.381 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-08-16T19:07:59.382 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-08-16T19:07:59.382 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:07:59.382 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 msgr2=0x7f41c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 0x7f41c00a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009230 tx=0x7f41bc0058d0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 msgr2=0x7f41d4100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c920 tx=0x7f41c400cde0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 shutdown_connections 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d410be90 0x7f41d4101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f41c00a2b80 0x7f41c00a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41d41033d0 0x7f41d4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 --2- 172.21.15.62:0/992376215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41d4102ab0 0x7f41d4100710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:07:59.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.382+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 >> 172.21.15.62:0/992376215 conn(0x7f41d40fc4b0 msgr2=0x7f41d40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:07:59.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.383+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 shutdown_connections 2024-08-16T19:07:59.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:07:59.383+0000 7f41d950e640 1 -- 172.21.15.62:0/992376215 wait complete. 2024-08-16T19:07:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:59 smithi155 ceph-mon[37413]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:07:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:59 smithi161 ceph-mon[37576]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:07:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:07:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:07:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.033 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-16T19:08:00.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-16T19:08:00.314 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:00.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/992376215' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-16T19:08:00.553 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.553 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.553 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:08:00.553 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:08:00.553 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/992376215' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:08:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/992376215' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:08:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:08:00.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 -- 172.21.15.62:0/3447305124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102250 msgr2=0x7fd62810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 --2- 172.21.15.62:0/3447305124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102250 0x7fd62810e780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd620009cd0 tx=0x7fd62002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 -- 172.21.15.62:0/3447305124 shutdown_connections 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 --2- 172.21.15.62:0/3447305124 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd628102250 0x7fd62810e780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 --2- 172.21.15.62:0/3447305124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6281018b0 0x7fd628101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 --2- 172.21.15.62:0/3447305124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281078b0 0x7fd628107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 -- 172.21.15.62:0/3447305124 >> 172.21.15.62:0/3447305124 conn(0x7fd6280fd5f0 msgr2=0x7fd6280ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 -- 172.21.15.62:0/3447305124 shutdown_connections 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.767+0000 7fd62c951640 1 -- 172.21.15.62:0/3447305124 wait complete. 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 Processor -- start 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 -- start start 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 0x7fd628074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 0x7fd628074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111cb0 con 0x7fd6281078b0 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111b30 con 0x7fd6281018b0 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62c951640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111e30 con 0x7fd628102250 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd626d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd626d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55586/0 (socket says 172.21.15.62:55586) 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd626d9d640 1 -- 172.21.15.62:0/1364450377 learned_addr learned my addr 172.21.15.62:0/1364450377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.768+0000 7fd62659c640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 0x7fd628074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd625d9b640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 0x7fd628074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 -- 172.21.15.62:0/1364450377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 msgr2=0x7fd628074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 0x7fd628074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 -- 172.21.15.62:0/1364450377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 msgr2=0x7fd628074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 0x7fd628074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 -- 172.21.15.62:0/1364450377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281aaaf0 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd62659c640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 0x7fd628074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd625d9b640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 0x7fd628074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd626d9d640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd620005cc0 tx=0x7fd620003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004040 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6200046c0 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281aad20 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620046480 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.769+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281ab200 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.770+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628102ff0 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.772+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6200305a0 con 0x7fd6281078b0 2024-08-16T19:08:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.773+0000 7fd61f7fe640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.773+0000 7fd62659c640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.773+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd62002e030 con 0x7fd6281078b0 2024-08-16T19:08:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.773+0000 7fd62659c640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 0x7fd6140a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd610005fd0 tx=0x7fd6100094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.776+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd62004b060 con 0x7fd6281078b0 2024-08-16T19:08:00.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:00.952+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd628101d10 con 0x7fd6281078b0 2024-08-16T19:08:01.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.436+0000 7fd61f7fe640 1 -- 172.21.15.62:0/1364450377 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fd628101d10 con 0x7fd6281078b0 2024-08-16T19:08:01.436 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-08-16T19:08:01.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 msgr2=0x7fd6140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:01.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 0x7fd6140a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd610005fd0 tx=0x7fd6100094d0 comp rx=0 tx=0).stop 2024-08-16T19:08:01.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 msgr2=0x7fd628074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:01.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd620005cc0 tx=0x7fd620003970 comp rx=0 tx=0).stop 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 shutdown_connections 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6281078b0 0x7fd628074ca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd628102250 0x7fd628074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6140a2b80 0x7fd6140a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 --2- 172.21.15.62:0/1364450377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6281018b0 0x7fd628074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 >> 172.21.15.62:0/1364450377 conn(0x7fd6280fd5f0 msgr2=0x7fd6280fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 shutdown_connections 2024-08-16T19:08:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:01.439+0000 7fd62c951640 1 -- 172.21.15.62:0/1364450377 wait complete. 2024-08-16T19:08:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:01 smithi062 ceph-mon[32993]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:01 smithi155 ceph-mon[37413]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:01 smithi161 ceph-mon[37576]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:02.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-16T19:08:02.455 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:02 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-16T19:08:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:02 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-16T19:08:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:02 smithi062 ceph-mon[32993]: osdmap e78: 12 total, 12 up, 12 in 2024-08-16T19:08:02.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:02 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-16T19:08:02.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:02 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-16T19:08:02.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:02 smithi155 ceph-mon[37413]: osdmap e78: 12 total, 12 up, 12 in 2024-08-16T19:08:02.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:02 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-16T19:08:02.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:02 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1364450377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-16T19:08:02.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:02 smithi161 ceph-mon[37576]: osdmap e78: 12 total, 12 up, 12 in 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 -- 172.21.15.62:0/3355313302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 msgr2=0x7f3cc0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 --2- 172.21.15.62:0/3355313302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0100f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009900 tx=0x7f3cb002a480 comp rx=0 tx=0).stop 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 -- 172.21.15.62:0/3355313302 shutdown_connections 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 --2- 172.21.15.62:0/3355313302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc0102c20 0x7f3cc010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 --2- 172.21.15.62:0/3355313302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc01022a0 0x7f3cc01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 --2- 172.21.15.62:0/3355313302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0100f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:02.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 -- 172.21.15.62:0/3355313302 >> 172.21.15.62:0/3355313302 conn(0x7f3cc00fc500 msgr2=0x7f3cc00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 -- 172.21.15.62:0/3355313302 shutdown_connections 2024-08-16T19:08:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 -- 172.21.15.62:0/3355313302 wait complete. 2024-08-16T19:08:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.930+0000 7f3cc622d640 1 Processor -- start 2024-08-16T19:08:02.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 -- start start 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc01022a0 0x7f3cc0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 0x7f3cc019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0111dd0 con 0x7f3cc0100b50 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0111c50 con 0x7f3cc01022a0 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cc622d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0111f50 con 0x7f3cc0102c20 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36768/0 (socket says 172.21.15.62:36768) 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 -- 172.21.15.62:0/2387529760 learned_addr learned my addr 172.21.15.62:0/2387529760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbeffd640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc01022a0 0x7f3cc0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbffff640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 0x7f3cc019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:02.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 -- 172.21.15.62:0/2387529760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 msgr2=0x7f3cc019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 0x7f3cc019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 -- 172.21.15.62:0/2387529760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc01022a0 msgr2=0x7f3cc0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc01022a0 0x7f3cc0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbf7fe640 1 -- 172.21.15.62:0/2387529760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc019d1e0 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.931+0000 7f3cbffff640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 0x7f3cc019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cbf7fe640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0006010 tx=0x7f3cb0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb003f070 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc019d470 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb0031e30 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0035030 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.932+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01a4db0 con 0x7f3cc0100b50 2024-08-16T19:08:02.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.933+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc0100f30 con 0x7f3cc0100b50 2024-08-16T19:08:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.934+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3cb0042070 con 0x7f3cc0100b50 2024-08-16T19:08:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.935+0000 7f3cbcff9640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 0x7f3ca80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.935+0000 7f3cbeffd640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 0x7f3ca80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:02.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.935+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f3cb00efd20 con 0x7f3cc0100b50 2024-08-16T19:08:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.936+0000 7f3cbeffd640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 0x7f3ca80a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00ffa60 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:02.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:02.939+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3cb00bcfa0 con 0x7f3cc0100b50 2024-08-16T19:08:03.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.120+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3cc01a4f80 con 0x7f3cc0100b50 2024-08-16T19:08:03.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.567+0000 7f3cbcff9640 1 -- 172.21.15.62:0/2387529760 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f3cb00bc8a0 con 0x7f3cc0100b50 2024-08-16T19:08:03.568 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 msgr2=0x7f3ca80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 0x7f3ca80a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00ffa60 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 msgr2=0x7f3cc0198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0006010 tx=0x7f3cb0003c80 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 shutdown_connections 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cc0102c20 0x7f3cc019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3ca80a2c00 0x7f3ca80a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cc01022a0 0x7f3cc0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 --2- 172.21.15.62:0/2387529760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cc0100b50 0x7f3cc0198230 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 >> 172.21.15.62:0/2387529760 conn(0x7f3cc00fc500 msgr2=0x7f3cc00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 shutdown_connections 2024-08-16T19:08:03.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:03.569+0000 7f3cc622d640 1 -- 172.21.15.62:0/2387529760 wait complete. 2024-08-16T19:08:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:03 smithi155 ceph-mon[37413]: pgmap v250: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:03 smithi155 ceph-mon[37413]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:03 smithi155 ceph-mon[37413]: osdmap e79: 12 total, 12 up, 12 in 2024-08-16T19:08:03.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:03 smithi062 ceph-mon[32993]: pgmap v250: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:03.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:03 smithi062 ceph-mon[32993]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:03.886 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:03 smithi062 ceph-mon[32993]: osdmap e79: 12 total, 12 up, 12 in 2024-08-16T19:08:03.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:03 smithi161 ceph-mon[37576]: pgmap v250: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:03.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:03 smithi161 ceph-mon[37576]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:03.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:03 smithi161 ceph-mon[37576]: osdmap e79: 12 total, 12 up, 12 in 2024-08-16T19:08:04.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-16T19:08:04.616 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:04.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:04 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-16T19:08:04.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:04 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-16T19:08:04.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:04 smithi155 ceph-mon[37413]: osdmap e80: 12 total, 12 up, 12 in 2024-08-16T19:08:04.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:04 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-16T19:08:04.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:04 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-16T19:08:04.850 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:04 smithi062 ceph-mon[32993]: osdmap e80: 12 total, 12 up, 12 in 2024-08-16T19:08:04.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:04 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-16T19:08:04.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:04 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2387529760' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-16T19:08:04.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:04 smithi161 ceph-mon[37576]: osdmap e80: 12 total, 12 up, 12 in 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 -- 172.21.15.62:0/814428533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7110102250 msgr2=0x7f711010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 --2- 172.21.15.62:0/814428533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7110102250 0x7f711010e780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7108009cd0 tx=0x7f710802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 -- 172.21.15.62:0/814428533 shutdown_connections 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 --2- 172.21.15.62:0/814428533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7110102250 0x7f711010e780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 --2- 172.21.15.62:0/814428533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 0x7f7110101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 --2- 172.21.15.62:0/814428533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71101078b0 0x7f7110107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 -- 172.21.15.62:0/814428533 >> 172.21.15.62:0/814428533 conn(0x7f71100fd5f0 msgr2=0x7f71100ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 -- 172.21.15.62:0/814428533 shutdown_connections 2024-08-16T19:08:05.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.134+0000 7f7115044640 1 -- 172.21.15.62:0/814428533 wait complete. 2024-08-16T19:08:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 Processor -- start 2024-08-16T19:08:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 -- start start 2024-08-16T19:08:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 0x7f71101982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:05.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7110102250 0x7f7110198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111d50 con 0x7f71101078b0 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111bd0 con 0x7f7110102250 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f7115044640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111ed0 con 0x7f71101018b0 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36796/0 (socket says 172.21.15.62:36796) 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 -- 172.21.15.62:0/237269730 learned_addr learned my addr 172.21.15.62:0/237269730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710e59c640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7110102250 0x7f7110198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710ed9d640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 0x7f71101982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 -- 172.21.15.62:0/237269730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 msgr2=0x7f71101982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 0x7f71101982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 -- 172.21.15.62:0/237269730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7110102250 msgr2=0x7f7110198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.135+0000 7f710f59e640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7110102250 0x7f7110198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f710f59e640 1 -- 172.21.15.62:0/237269730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711019d280 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f710f59e640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7108006010 tx=0x7f7108003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108004450 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7108003e10 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711019d510 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108046480 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.136+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71101a4e50 con 0x7f71101078b0 2024-08-16T19:08:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.137+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7110101d90 con 0x7f71101078b0 2024-08-16T19:08:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.138+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71080045f0 con 0x7f71101078b0 2024-08-16T19:08:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.139+0000 7f70f7fff640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.139+0000 7f710ed9d640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.139+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f710802e030 con 0x7f71101078b0 2024-08-16T19:08:05.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.139+0000 7f710ed9d640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 0x7f70f80a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7100009890 tx=0x7f7100008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:05.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.142+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f710804b060 con 0x7f71101078b0 2024-08-16T19:08:05.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.355+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7110199630 con 0x7f71101078b0 2024-08-16T19:08:05.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.575+0000 7f70f7fff640 1 -- 172.21.15.62:0/237269730 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7110199630 con 0x7f71101078b0 2024-08-16T19:08:05.576 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-08-16T19:08:05.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 msgr2=0x7f70f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 0x7f70f80a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7100009890 tx=0x7f7100008040 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 msgr2=0x7f711019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7108006010 tx=0x7f7108003c20 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 shutdown_connections 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71101078b0 0x7f711019cba0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f70f80a2b80 0x7f70f80a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7110102250 0x7f7110198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 --2- 172.21.15.62:0/237269730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71101018b0 0x7f71101982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 >> 172.21.15.62:0/237269730 conn(0x7f71100fd5f0 msgr2=0x7f71100fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 shutdown_connections 2024-08-16T19:08:05.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:05.578+0000 7f7115044640 1 -- 172.21.15.62:0/237269730 wait complete. 2024-08-16T19:08:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:05 smithi161 ceph-mon[37576]: pgmap v253: 97 pgs: 1 creating+activating, 6 creating+peering, 85 unknown, 5 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:05 smithi161 ceph-mon[37576]: osdmap e81: 12 total, 12 up, 12 in 2024-08-16T19:08:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:05 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-16T19:08:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:05 smithi161 ceph-mon[37576]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-16T19:08:05.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:05 smithi062 ceph-mon[32993]: pgmap v253: 97 pgs: 1 creating+activating, 6 creating+peering, 85 unknown, 5 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:05.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:05 smithi062 ceph-mon[32993]: osdmap e81: 12 total, 12 up, 12 in 2024-08-16T19:08:05.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:05 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-16T19:08:05.955 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:05 smithi062 ceph-mon[32993]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-16T19:08:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:05 smithi155 ceph-mon[37413]: pgmap v253: 97 pgs: 1 creating+activating, 6 creating+peering, 85 unknown, 5 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:05 smithi155 ceph-mon[37413]: osdmap e81: 12 total, 12 up, 12 in 2024-08-16T19:08:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:05 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-16T19:08:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:05 smithi155 ceph-mon[37413]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-16T19:08:06.331 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-16T19:08:06.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: osdmap e82: 12 total, 12 up, 12 in 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:active 2024-08-16T19:08:06.589 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:06 smithi062 ceph-mon[32993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:06.602 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: osdmap e82: 12 total, 12 up, 12 in 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:active 2024-08-16T19:08:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:06 smithi161 ceph-mon[37576]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/237269730' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: osdmap e82: 12 total, 12 up, 12 in 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] up:active 2024-08-16T19:08:07.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:06 smithi155 ceph-mon[37413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 -- 172.21.15.62:0/1511866968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d8102250 msgr2=0x7f68d810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 --2- 172.21.15.62:0/1511866968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d8102250 0x7f68d810e780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009cd0 tx=0x7f68d002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 -- 172.21.15.62:0/1511866968 shutdown_connections 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 --2- 172.21.15.62:0/1511866968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d8102250 0x7f68d810e780 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 --2- 172.21.15.62:0/1511866968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d81018b0 0x7f68d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 --2- 172.21.15.62:0/1511866968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81078b0 0x7f68d8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 -- 172.21.15.62:0/1511866968 >> 172.21.15.62:0/1511866968 conn(0x7f68d80fd5f0 msgr2=0x7f68d80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 -- 172.21.15.62:0/1511866968 shutdown_connections 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.080+0000 7f68dd465640 1 -- 172.21.15.62:0/1511866968 wait complete. 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 Processor -- start 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 -- start start 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 0x7f68d8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 0x7f68d8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8111d00 con 0x7f68d81078b0 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8111b80 con 0x7f68d81018b0 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68dd465640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8111e80 con 0x7f68d8102250 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d6ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 0x7f68d8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36816/0 (socket says 172.21.15.62:36816) 2024-08-16T19:08:07.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 -- 172.21.15.62:0/1918118400 learned_addr learned my addr 172.21.15.62:0/1918118400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68cffff640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 0x7f68d8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 -- 172.21.15.62:0/1918118400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 msgr2=0x7f68d8074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 0x7f68d8074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 -- 172.21.15.62:0/1918118400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 msgr2=0x7f68d8074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 0x7f68d8074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d77fe640 1 -- 172.21.15.62:0/1918118400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d81a6660 con 0x7f68d81078b0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68cffff640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 0x7f68d8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.081+0000 7f68d6ffd640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 0x7f68d8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68d77fe640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68d0033040 tx=0x7f68d0031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0004040 con 0x7f68d81078b0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d00046c0 con 0x7f68d81078b0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0046480 con 0x7f68d81078b0 2024-08-16T19:08:07.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d81a68f0 con 0x7f68d81078b0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.082+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a6e80 con 0x7f68d81078b0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.083+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d8101d10 con 0x7f68d81078b0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.084+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68d00305a0 con 0x7f68d81078b0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.085+0000 7f68d4ff9640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 0x7f68c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.085+0000 7f68d6ffd640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 0x7f68c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.085+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f68d002e030 con 0x7f68d81078b0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.085+0000 7f68d6ffd640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 0x7f68c40a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f40 tx=0x7f68c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:07.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.088+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68d004b060 con 0x7f68d81078b0 2024-08-16T19:08:07.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.272+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f68d8075750 con 0x7f68d81078b0 2024-08-16T19:08:07.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.596+0000 7f68d4ff9640 1 -- 172.21.15.62:0/1918118400 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f68d8075750 con 0x7f68d81078b0 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 msgr2=0x7f68c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 0x7f68c40a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009f40 tx=0x7f68c8005f90 comp rx=0 tx=0).stop 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 msgr2=0x7f68d8074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68d0033040 tx=0x7f68d0031e60 comp rx=0 tx=0).stop 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 shutdown_connections 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68d81078b0 0x7f68d8074ca0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68d8102250 0x7f68d8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68c40a2c00 0x7f68c40a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 --2- 172.21.15.62:0/1918118400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68d81018b0 0x7f68d8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:07.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.598+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 >> 172.21.15.62:0/1918118400 conn(0x7f68d80fd5f0 msgr2=0x7f68d80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:07.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.599+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 shutdown_connections 2024-08-16T19:08:07.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:07.599+0000 7f68dd465640 1 -- 172.21.15.62:0/1918118400 wait complete. 2024-08-16T19:08:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:07 smithi161 ceph-mon[37576]: pgmap v256: 97 pgs: 1 creating+activating, 6 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:07 smithi161 ceph-mon[37576]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-16T19:08:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:07 smithi161 ceph-mon[37576]: Cluster is now healthy 2024-08-16T19:08:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:07 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-16T19:08:07.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:07 smithi062 ceph-mon[32993]: pgmap v256: 97 pgs: 1 creating+activating, 6 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:07.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:07 smithi062 ceph-mon[32993]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-16T19:08:07.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:07 smithi062 ceph-mon[32993]: Cluster is now healthy 2024-08-16T19:08:07.923 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:07 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-16T19:08:07.932 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:07 smithi155 ceph-mon[37413]: pgmap v256: 97 pgs: 1 creating+activating, 6 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:07.932 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:07 smithi155 ceph-mon[37413]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-16T19:08:07.932 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:07 smithi155 ceph-mon[37413]: Cluster is now healthy 2024-08-16T19:08:07.932 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:07 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-16T19:08:08.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-16T19:08:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:08.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:08 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-16T19:08:08.852 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:08 smithi062 ceph-mon[32993]: osdmap e83: 12 total, 12 up, 12 in 2024-08-16T19:08:08.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:08 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-16T19:08:08.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:08 smithi161 ceph-mon[37576]: osdmap e83: 12 total, 12 up, 12 in 2024-08-16T19:08:09.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:08 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1918118400' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-16T19:08:09.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:08 smithi155 ceph-mon[37413]: osdmap e83: 12 total, 12 up, 12 in 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 -- 172.21.15.62:0/1837211971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81067b0 msgr2=0x7f46a8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 --2- 172.21.15.62:0/1837211971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81067b0 0x7f46a8106b90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009930 tx=0x7f46a402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 -- 172.21.15.62:0/1837211971 shutdown_connections 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 --2- 172.21.15.62:0/1837211971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a8101150 0x7f46a810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 --2- 172.21.15.62:0/1837211971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 --2- 172.21.15.62:0/1837211971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a81067b0 0x7f46a8106b90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 -- 172.21.15.62:0/1837211971 >> 172.21.15.62:0/1837211971 conn(0x7f46a80fc4d0 msgr2=0x7f46a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 -- 172.21.15.62:0/1837211971 shutdown_connections 2024-08-16T19:08:09.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.075+0000 7f46af30a640 1 -- 172.21.15.62:0/1837211971 wait complete. 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 Processor -- start 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 -- start start 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a81067b0 0x7f46a819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8110c50 con 0x7f46a8101150 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8110ad0 con 0x7f46a81007b0 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46af30a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8110dd0 con 0x7f46a81067b0 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46ac8a5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46ac8a5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36842/0 (socket says 172.21.15.62:36842) 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46ac8a5640 1 -- 172.21.15.62:0/2096735388 learned_addr learned my addr 172.21.15.62:0/2096735388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:09.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.076+0000 7f46ad0a6640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 -- 172.21.15.62:0/2096735388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a81067b0 msgr2=0x7f46a819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a81067b0 0x7f46a819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 -- 172.21.15.62:0/2096735388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 msgr2=0x7f46a8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 -- 172.21.15.62:0/2096735388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a819d2f0 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ad0a6640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46ac8a5640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f469800b4a0 tx=0x7f469800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698014070 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4698010af0 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a819d5e0 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698017d10 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.077+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a81a4ec0 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.078+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a8100c90 con 0x7f46a8101150 2024-08-16T19:08:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.079+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4698017070 con 0x7f46a8101150 2024-08-16T19:08:09.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.080+0000 7f469e7fc640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 0x7f46940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:09.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.080+0000 7f46ad0a6640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 0x7f46940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:09.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.080+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f46980ce830 con 0x7f46a8101150 2024-08-16T19:08:09.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.080+0000 7f46ad0a6640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 0x7f46940a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46a4000c00 tx=0x7f46a4032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:09.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.083+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f469809b760 con 0x7f46a8101150 2024-08-16T19:08:09.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:09.259+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f46a802d5e0 con 0x7f46a8101150 2024-08-16T19:08:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:10 smithi155 ceph-mon[37413]: pgmap v258: 97 pgs: 8 unknown, 89 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-08-16T19:08:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:10 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-16T19:08:10.329 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:08:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:08:10.044+0000 7f48faf90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-16T19:08:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:10 smithi161 ceph-mon[37576]: pgmap v258: 97 pgs: 8 unknown, 89 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-08-16T19:08:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:10 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-16T19:08:10.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:10 smithi062 ceph-mon[32993]: pgmap v258: 97 pgs: 8 unknown, 89 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-08-16T19:08:10.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:10 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-16T19:08:10.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.608+0000 7f469e7fc640 1 -- 172.21.15.62:0/2096735388 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f469809b060 con 0x7f46a8101150 2024-08-16T19:08:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data_ec' created 2024-08-16T19:08:10.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 msgr2=0x7f46940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:10.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 0x7f46940a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46a4000c00 tx=0x7f46a4032d50 comp rx=0 tx=0).stop 2024-08-16T19:08:10.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 msgr2=0x7f46a8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:10.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f469800b4a0 tx=0x7f469800b960 comp rx=0 tx=0).stop 2024-08-16T19:08:10.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 shutdown_connections 2024-08-16T19:08:10.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46a81067b0 0x7f46a819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46a8101150 0x7f46a8198880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f46940a2c00 0x7f46940a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 --2- 172.21.15.62:0/2096735388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46a81007b0 0x7f46a8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.611+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 >> 172.21.15.62:0/2096735388 conn(0x7f46a80fc4d0 msgr2=0x7f46a806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.612+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 shutdown_connections 2024-08-16T19:08:10.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:10.612+0000 7f46af30a640 1 -- 172.21.15.62:0/2096735388 wait complete. 2024-08-16T19:08:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:11 smithi062 ceph-mon[32993]: pgmap v260: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-08-16T19:08:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:11 smithi062 ceph-mon[32993]: osdmap e84: 12 total, 12 up, 12 in 2024-08-16T19:08:11.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:11 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-16T19:08:11.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:11 smithi062 ceph-mon[32993]: osdmap e85: 12 total, 12 up, 12 in 2024-08-16T19:08:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:11 smithi155 ceph-mon[37413]: pgmap v260: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-08-16T19:08:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:11 smithi155 ceph-mon[37413]: osdmap e84: 12 total, 12 up, 12 in 2024-08-16T19:08:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:11 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-16T19:08:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:11 smithi155 ceph-mon[37413]: osdmap e85: 12 total, 12 up, 12 in 2024-08-16T19:08:11.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-16T19:08:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:11 smithi161 ceph-mon[37576]: pgmap v260: 97 pgs: 97 active+clean; 578 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 7 op/s 2024-08-16T19:08:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:11 smithi161 ceph-mon[37576]: osdmap e84: 12 total, 12 up, 12 in 2024-08-16T19:08:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:11 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2096735388' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-16T19:08:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:11 smithi161 ceph-mon[37576]: osdmap e85: 12 total, 12 up, 12 in 2024-08-16T19:08:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/3760642554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e41067b0 msgr2=0x7fe5e4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/3760642554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e41067b0 0x7fe5e4106b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009930 tx=0x7fe5e002a420 comp rx=0 tx=0).stop 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/3760642554 shutdown_connections 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/3760642554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e4101150 0x7fe5e410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/3760642554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/3760642554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e41067b0 0x7fe5e4106b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/3760642554 >> 172.21.15.62:0/3760642554 conn(0x7fe5e40fc4d0 msgr2=0x7fe5e40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/3760642554 shutdown_connections 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.063+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/3760642554 wait complete. 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 Processor -- start 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 -- start start 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 0x7fe5e419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:12.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4110bf0 con 0x7fe5e4101150 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4110a70 con 0x7fe5e41067b0 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5eb0d4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4110d70 con 0x7fe5e41007b0 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59096/0 (socket says 172.21.15.62:59096) 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5dbfff640 1 -- 172.21.15.62:0/1402524486 learned_addr learned my addr 172.21.15.62:0/1402524486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5e8e70640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.064+0000 7fe5e9671640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 0x7fe5e419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 -- 172.21.15.62:0/1402524486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 msgr2=0x7fe5e4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 -- 172.21.15.62:0/1402524486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 msgr2=0x7fe5e419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 0x7fe5e419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 -- 172.21.15.62:0/1402524486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e419d1d0 con 0x7fe5e4101150 2024-08-16T19:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5e8e70640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5e9671640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 0x7fe5e419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5dbfff640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d770 tx=0x7fe5d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4017070 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d4005500 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4004e00 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e419d4c0 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.065+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a4da0 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.066+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e4100c10 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.067+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5d4004fa0 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.068+0000 7fe5d9ffb640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 0x7fe5cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.068+0000 7fe5e8e70640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 0x7fe5cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.068+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7fe5d40ced90 con 0x7fe5e4101150 2024-08-16T19:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.068+0000 7fe5e8e70640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 0x7fe5cc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000c00 tx=0x7fe5e0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:12.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.072+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe5d409bad0 con 0x7fe5e4101150 2024-08-16T19:08:12.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.247+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe5e41a4f70 con 0x7fe5e4101150 2024-08-16T19:08:12.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.616+0000 7fe5d9ffb640 1 -- 172.21.15.62:0/1402524486 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7fe5d409b3d0 con 0x7fe5e4101150 2024-08-16T19:08:12.617 INFO:teuthology.orchestra.run.smithi062.stderr:set pool 4 allow_ec_overwrites to true 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 msgr2=0x7fe5cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 0x7fe5cc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0000c00 tx=0x7fe5e0032c70 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 msgr2=0x7fe5e4198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d770 tx=0x7fe5d400dc30 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 shutdown_connections 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe5cc0a2b80 0x7fe5cc0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e41067b0 0x7fe5e419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5e4101150 0x7fe5e4198760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 --2- 172.21.15.62:0/1402524486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5e41007b0 0x7fe5e4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 >> 172.21.15.62:0/1402524486 conn(0x7fe5e40fc4d0 msgr2=0x7fe5e4104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 shutdown_connections 2024-08-16T19:08:12.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:12.619+0000 7fe5eb0d4640 1 -- 172.21.15.62:0/1402524486 wait complete. 2024-08-16T19:08:12.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:12 smithi161 ceph-mon[37576]: osdmap e86: 12 total, 12 up, 12 in 2024-08-16T19:08:12.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:12 smithi161 ceph-mon[37576]: pgmap v263: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 252 B/s wr, 1 op/s 2024-08-16T19:08:12.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:12 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-16T19:08:12.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:12 smithi062 ceph-mon[32993]: osdmap e86: 12 total, 12 up, 12 in 2024-08-16T19:08:12.941 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:12 smithi062 ceph-mon[32993]: pgmap v263: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 252 B/s wr, 1 op/s 2024-08-16T19:08:12.942 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:12 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-16T19:08:13.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:12 smithi155 ceph-mon[37413]: osdmap e86: 12 total, 12 up, 12 in 2024-08-16T19:08:13.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:12 smithi155 ceph-mon[37413]: pgmap v263: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 252 B/s wr, 1 op/s 2024-08-16T19:08:13.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:12 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-16T19:08:13.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-16T19:08:13.682 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:13 smithi062 ceph-mon[32993]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:13 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-16T19:08:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:13 smithi062 ceph-mon[32993]: osdmap e87: 12 total, 12 up, 12 in 2024-08-16T19:08:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:13 smithi161 ceph-mon[37576]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:13 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-16T19:08:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:13 smithi161 ceph-mon[37576]: osdmap e87: 12 total, 12 up, 12 in 2024-08-16T19:08:14.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:13 smithi155 ceph-mon[37413]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-16T19:08:14.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:13 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1402524486' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-16T19:08:14.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:13 smithi155 ceph-mon[37413]: osdmap e87: 12 total, 12 up, 12 in 2024-08-16T19:08:14.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 -- 172.21.15.62:0/1643090376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c10bea0 msgr2=0x7fab8c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:14.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 --2- 172.21.15.62:0/1643090376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c10bea0 0x7fab8c10e340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fab84009ca0 tx=0x7fab8402a950 comp rx=0 tx=0).stop 2024-08-16T19:08:14.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 -- 172.21.15.62:0/1643090376 shutdown_connections 2024-08-16T19:08:14.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 --2- 172.21.15.62:0/1643090376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c10bea0 0x7fab8c10e340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 --2- 172.21.15.62:0/1643090376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c103390 0x7fab8c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 --2- 172.21.15.62:0/1643090376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c102a70 0x7fab8c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 -- 172.21.15.62:0/1643090376 >> 172.21.15.62:0/1643090376 conn(0x7fab8c0fc4d0 msgr2=0x7fab8c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:14.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 -- 172.21.15.62:0/1643090376 shutdown_connections 2024-08-16T19:08:14.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.122+0000 7fab94183640 1 -- 172.21.15.62:0/1643090376 wait complete. 2024-08-16T19:08:14.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 Processor -- start 2024-08-16T19:08:14.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 -- start start 2024-08-16T19:08:14.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c102a70 0x7fab8c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 0x7fab8c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111c80 con 0x7fab8c103390 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111b00 con 0x7fab8c10bea0 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab94183640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111e00 con 0x7fab8c102a70 2024-08-16T19:08:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59112/0 (socket says 172.21.15.62:59112) 2024-08-16T19:08:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 -- 172.21.15.62:0/948817651 learned_addr learned my addr 172.21.15.62:0/948817651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab91f1f640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c102a70 0x7fab8c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab92720640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 0x7fab8c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:14.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 -- 172.21.15.62:0/948817651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c102a70 msgr2=0x7fab8c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c102a70 0x7fab8c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 -- 172.21.15.62:0/948817651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 msgr2=0x7fab8c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 0x7fab8c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab9171e640 1 -- 172.21.15.62:0/948817651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c19d140 con 0x7fab8c103390 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.123+0000 7fab92720640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 0x7fab8c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab9171e640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00d750 tx=0x7fab7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c017070 con 0x7fab8c103390 2024-08-16T19:08:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c19d430 con 0x7fab8c103390 2024-08-16T19:08:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab7c005020 con 0x7fab8c103390 2024-08-16T19:08:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c004d00 con 0x7fab8c103390 2024-08-16T19:08:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.124+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a4d10 con 0x7fab8c103390 2024-08-16T19:08:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.125+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c102e50 con 0x7fab8c103390 2024-08-16T19:08:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.126+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab7c009a30 con 0x7fab8c103390 2024-08-16T19:08:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.127+0000 7fab82ffd640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 0x7fab780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.127+0000 7fab91f1f640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 0x7fab780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.127+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fab7c0cf0c0 con 0x7fab8c103390 2024-08-16T19:08:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.127+0000 7fab91f1f640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 0x7fab780a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab880058c0 tx=0x7fab8800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.130+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fab7c09be00 con 0x7fab8c103390 2024-08-16T19:08:14.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.348+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fab8c106f10 con 0x7fab8c103390 2024-08-16T19:08:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.629+0000 7fab82ffd640 1 -- 172.21.15.62:0/948817651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fab7c09b700 con 0x7fab8c103390 2024-08-16T19:08:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:added data pool 4 to fsmap 2024-08-16T19:08:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 msgr2=0x7fab780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 0x7fab780a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fab880058c0 tx=0x7fab8800a4c0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 msgr2=0x7fab8c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00d750 tx=0x7fab7c00dc10 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 shutdown_connections 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fab780a2b80 0x7fab780a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c10bea0 0x7fab8c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab8c103390 0x7fab8c1986d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 --2- 172.21.15.62:0/948817651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab8c102a70 0x7fab8c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 >> 172.21.15.62:0/948817651 conn(0x7fab8c0fc4d0 msgr2=0x7fab8c0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 shutdown_connections 2024-08-16T19:08:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:14.632+0000 7fab94183640 1 -- 172.21.15.62:0/948817651 wait complete. 2024-08-16T19:08:14.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:14 smithi161 ceph-mon[37576]: pgmap v265: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 253 B/s wr, 1 op/s 2024-08-16T19:08:14.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:14 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-16T19:08:14.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:14 smithi062 ceph-mon[32993]: pgmap v265: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 253 B/s wr, 1 op/s 2024-08-16T19:08:14.950 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:14 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-16T19:08:15.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:14 smithi155 ceph-mon[37413]: pgmap v265: 129 pgs: 5 creating+peering, 18 unknown, 106 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 253 B/s wr, 1 op/s 2024-08-16T19:08:15.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:14 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-16T19:08:15.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:08:15.621 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:15 smithi062 ceph-mon[32993]: osdmap e88: 12 total, 12 up, 12 in 2024-08-16T19:08:15.637 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:15.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:15 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-16T19:08:15.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:15 smithi062 ceph-mon[32993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:15 smithi161 ceph-mon[37576]: osdmap e88: 12 total, 12 up, 12 in 2024-08-16T19:08:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:15 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-16T19:08:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:15 smithi161 ceph-mon[37576]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:16.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:15 smithi155 ceph-mon[37413]: osdmap e88: 12 total, 12 up, 12 in 2024-08-16T19:08:16.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:15 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/948817651' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-16T19:08:16.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:15 smithi155 ceph-mon[37413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:16.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 -- 172.21.15.62:0/3643317963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481022a0 msgr2=0x7f9f481026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/3643317963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481022a0 0x7f9f481026e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c02a480 comp rx=0 tx=0).stop 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 -- 172.21.15.62:0/3643317963 shutdown_connections 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/3643317963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f48102c20 0x7f9f4810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/3643317963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f481022a0 0x7f9f481026e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/3643317963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100b50 0x7f9f48100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 -- 172.21.15.62:0/3643317963 >> 172.21.15.62:0/3643317963 conn(0x7f9f480fc500 msgr2=0x7f9f480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 -- 172.21.15.62:0/3643317963 shutdown_connections 2024-08-16T19:08:16.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.096+0000 7f9f4cff6640 1 -- 172.21.15.62:0/3643317963 wait complete. 2024-08-16T19:08:16.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 Processor -- start 2024-08-16T19:08:16.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 -- start start 2024-08-16T19:08:16.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:16.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f481022a0 0x7f9f481977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:16.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 0x7f9f4819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48110d50 con 0x7f9f48100b50 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48110bd0 con 0x7f9f48102c20 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4cff6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48110ed0 con 0x7f9f481022a0 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59124/0 (socket says 172.21.15.62:59124) 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4659c640 1 -- 172.21.15.62:0/2629171278 learned_addr learned my addr 172.21.15.62:0/2629171278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f46d9d640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 0x7f9f4819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f45d9b640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f481022a0 0x7f9f481977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.097+0000 7f9f4659c640 1 -- 172.21.15.62:0/2629171278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f481022a0 msgr2=0x7f9f481977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4659c640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f481022a0 0x7f9f481977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4659c640 1 -- 172.21.15.62:0/2629171278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 msgr2=0x7f9f4819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4659c640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 0x7f9f4819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4659c640 1 -- 172.21.15.62:0/2629171278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4819c220 con 0x7f9f48100b50 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f46d9d640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 0x7f9f4819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4659c640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b4a0 tx=0x7f9f4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40014070 con 0x7f9f48100b50 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40010af0 con 0x7f9f48100b50 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4819c510 con 0x7f9f48100b50 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017d10 con 0x7f9f48100b50 2024-08-16T19:08:16.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.098+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f48069ca0 con 0x7f9f48100b50 2024-08-16T19:08:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.099+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f48100fb0 con 0x7f9f48100b50 2024-08-16T19:08:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.100+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9f40017070 con 0x7f9f48100b50 2024-08-16T19:08:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.101+0000 7f9f377fe640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 0x7f9f300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.101+0000 7f9f45d9b640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 0x7f9f300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.101+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f9f400ce700 con 0x7f9f48100b50 2024-08-16T19:08:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.101+0000 7f9f45d9b640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 0x7f9f300a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0096c0 tx=0x7f9f3c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:16.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.105+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9f4009b420 con 0x7f9f48100b50 2024-08-16T19:08:16.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.319+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f481985d0 con 0x7f9f48100b50 2024-08-16T19:08:16.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.320+0000 7f9f377fe640 1 -- 172.21.15.62:0/2629171278 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11854 (secure 0 0 0) 0x7f9f4009ad20 con 0x7f9f48100b50 2024-08-16T19:08:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:16.321 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"btime":"2024-08-16T19:08:14:618694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:14.349923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24476,"qdb_cluster":[24476]},"id":1}]} 2024-08-16T19:08:16.321 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-08-16T19:08:16.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 msgr2=0x7f9f300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:16.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 0x7f9f300a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0096c0 tx=0x7f9f3c000bc0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 msgr2=0x7f9f48197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b4a0 tx=0x7f9f4000b960 comp rx=0 tx=0).stop 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 shutdown_connections 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9f300a2b80 0x7f9f300a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48102c20 0x7f9f4819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f481022a0 0x7f9f481977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 --2- 172.21.15.62:0/2629171278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f48100b50 0x7f9f48197270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 >> 172.21.15.62:0/2629171278 conn(0x7f9f480fc500 msgr2=0x7f9f480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 shutdown_connections 2024-08-16T19:08:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:16.322+0000 7f9f4cff6640 1 -- 172.21.15.62:0/2629171278 wait complete. 2024-08-16T19:08:16.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:16 smithi161 ceph-mon[37576]: pgmap v267: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:16.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:16 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2629171278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:16.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:16 smithi062 ceph-mon[32993]: pgmap v267: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:16.944 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:16 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2629171278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:16.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:17.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:16 smithi155 ceph-mon[37413]: pgmap v267: 129 pgs: 5 creating+peering, 124 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:17.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:16 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2629171278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:17.244 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 -- 172.21.15.62:0/3973199675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 msgr2=0x7ffab4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 --2- 172.21.15.62:0/3973199675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab4100f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009900 tx=0x7ffaa402a480 comp rx=0 tx=0).stop 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 -- 172.21.15.62:0/3973199675 shutdown_connections 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 --2- 172.21.15.62:0/3973199675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 --2- 172.21.15.62:0/3973199675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 --2- 172.21.15.62:0/3973199675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab4100f30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 -- 172.21.15.62:0/3973199675 >> 172.21.15.62:0/3973199675 conn(0x7ffab40fc500 msgr2=0x7ffab40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 -- 172.21.15.62:0/3973199675 shutdown_connections 2024-08-16T19:08:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.700+0000 7ffaba182640 1 -- 172.21.15.62:0/3973199675 wait complete. 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 Processor -- start 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 -- start start 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4110cd0 con 0x7ffab4100b50 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4110b50 con 0x7ffab4102c20 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffaba182640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4110e50 con 0x7ffab41022a0 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffab37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffab37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59142/0 (socket says 172.21.15.62:59142) 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffab37fe640 1 -- 172.21.15.62:0/3318227380 learned_addr learned my addr 172.21.15.62:0/3318227380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.701+0000 7ffab3fff640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab2ffd640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 -- 172.21.15.62:0/3318227380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 msgr2=0x7ffab419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 -- 172.21.15.62:0/3318227380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 msgr2=0x7ffab419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 -- 172.21.15.62:0/3318227380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab419f390 con 0x7ffab4100b50 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab3fff640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab2ffd640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab37fe640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4006010 tx=0x7ffaa4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa403f070 con 0x7ffab4100b50 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4031e30 con 0x7ffab4100b50 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab419f620 con 0x7ffab4100b50 2024-08-16T19:08:17.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4035030 con 0x7ffab4100b50 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.702+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a6f60 con 0x7ffab4100b50 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.703+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab4101ee0 con 0x7ffab4100b50 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.704+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffaa4042070 con 0x7ffab4100b50 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.705+0000 7ffab0ff9640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 0x7ffa9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.705+0000 7ffab2ffd640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 0x7ffa9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.705+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ffaa40f0270 con 0x7ffab4100b50 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.705+0000 7ffab2ffd640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 0x7ffa9c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006fd0 tx=0x7ffaa8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:17.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.708+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffaa40bd020 con 0x7ffab4100b50 2024-08-16T19:08:17.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:17 smithi155 ceph-mon[37413]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-16T19:08:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:17 smithi155 ceph-mon[37413]: Cluster is now healthy 2024-08-16T19:08:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.884+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffab41a7130 con 0x7ffab4100b50 2024-08-16T19:08:17.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:17 smithi062 ceph-mon[32993]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-16T19:08:17.885 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:17 smithi062 ceph-mon[32993]: Cluster is now healthy 2024-08-16T19:08:17.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.886+0000 7ffab0ff9640 1 -- 172.21.15.62:0/3318227380 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24104 (secure 0 0 0) 0x7ffaa40bc920 con 0x7ffab4100b50 2024-08-16T19:08:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:17.887 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:14.349723+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:08:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:17 smithi161 ceph-mon[37576]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-16T19:08:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:17 smithi161 ceph-mon[37576]: Cluster is now healthy 2024-08-16T19:08:17.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 msgr2=0x7ffa9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 0x7ffa9c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006fd0 tx=0x7ffaa8008040 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 msgr2=0x7ffab419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4006010 tx=0x7ffaa4003c80 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 shutdown_connections 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa9c0a2b80 0x7ffa9c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffab4102c20 0x7ffab419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffab41022a0 0x7ffab419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 --2- 172.21.15.62:0/3318227380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab4100b50 0x7ffab419a3e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 >> 172.21.15.62:0/3318227380 conn(0x7ffab40fc500 msgr2=0x7ffab406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 shutdown_connections 2024-08-16T19:08:17.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:17.888+0000 7ffaba182640 1 -- 172.21.15.62:0/3318227380 wait complete. 2024-08-16T19:08:18.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:18.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:18 smithi062 ceph-mon[32993]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:18.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:18 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3318227380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:18.789 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:18 smithi161 ceph-mon[37576]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:18 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3318227380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:19.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:18 smithi155 ceph-mon[37413]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:19.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:18 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3318227380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:19.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 -- 172.21.15.62:0/3485323717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41078c0 msgr2=0x7f35f4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:19.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 --2- 172.21.15.62:0/3485323717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41078c0 0x7f35f4107ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009930 tx=0x7f35e402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:19.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 -- 172.21.15.62:0/3485323717 shutdown_connections 2024-08-16T19:08:19.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 --2- 172.21.15.62:0/3485323717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 0x7f35f410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 --2- 172.21.15.62:0/3485323717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41018c0 0x7f35f4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 --2- 172.21.15.62:0/3485323717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41078c0 0x7f35f4107ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 -- 172.21.15.62:0/3485323717 >> 172.21.15.62:0/3485323717 conn(0x7f35f40fd5e0 msgr2=0x7f35f40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 -- 172.21.15.62:0/3485323717 shutdown_connections 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.204+0000 7f35f9439640 1 -- 172.21.15.62:0/3485323717 wait complete. 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 Processor -- start 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 -- start start 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 0x7f35f419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41078c0 0x7f35f419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:19.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35f4111ce0 con 0x7f35f41018c0 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f35f4111b60 con 0x7f35f4102260 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f9439640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35f4111e60 con 0x7f35f41078c0 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59156/0 (socket says 172.21.15.62:59156) 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f2ffd640 1 -- 172.21.15.62:0/2140181488 learned_addr learned my addr 172.21.15.62:0/2140181488 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f27fc640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 0x7f35f419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.205+0000 7f35f37fe640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41078c0 0x7f35f419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 -- 172.21.15.62:0/2140181488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41078c0 msgr2=0x7f35f419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41078c0 0x7f35f419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 -- 172.21.15.62:0/2140181488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 msgr2=0x7f35f419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 0x7f35f419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 -- 172.21.15.62:0/2140181488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f419f3a0 con 0x7f35f41018c0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f2ffd640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009230 tx=0x7f35e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e403f030 con 0x7f35f41018c0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e4003ad0 con 0x7f35f41018c0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f419f630 con 0x7f35f41018c0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4035070 con 0x7f35f41018c0 2024-08-16T19:08:19.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.206+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f41a6f70 con 0x7f35f41018c0 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.207+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f4103000 con 0x7f35f41018c0 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.208+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f35e4042030 con 0x7f35f41018c0 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.209+0000 7f35ebfff640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 0x7f35e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.209+0000 7f35f27fc640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 0x7f35e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.209+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f35e40efbb0 con 0x7f35f41018c0 2024-08-16T19:08:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.210+0000 7f35f27fc640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 0x7f35e00a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35f410ec20 tx=0x7f35dc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:19.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.212+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f35e40bc8d0 con 0x7f35f41018c0 2024-08-16T19:08:19.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.388+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35f41a7140 con 0x7f35f41018c0 2024-08-16T19:08:19.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.390+0000 7f35ebfff640 1 -- 172.21.15.62:0/2140181488 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24104 (secure 0 0 0) 0x7f35e40bc1d0 con 0x7f35f41018c0 2024-08-16T19:08:19.391 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:19.391 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:14.349723+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 msgr2=0x7f35e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 0x7f35e00a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35f410ec20 tx=0x7f35dc00d040 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 msgr2=0x7f35f419a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009230 tx=0x7f35e4004270 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 shutdown_connections 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35f41078c0 0x7f35f419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f35e00a2b80 0x7f35e00a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35f4102260 0x7f35f419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 --2- 172.21.15.62:0/2140181488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35f41018c0 0x7f35f419a3f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 >> 172.21.15.62:0/2140181488 conn(0x7f35f40fd5e0 msgr2=0x7f35f40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 shutdown_connections 2024-08-16T19:08:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:19.392+0000 7f35f9439640 1 -- 172.21.15.62:0/2140181488 wait complete. 2024-08-16T19:08:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:19 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2140181488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:20.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:08:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:19 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2140181488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:20.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:19 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2140181488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:20.313 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:20.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:20 smithi062 ceph-mon[32993]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/240690501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 msgr2=0x7ffba4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/240690501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba4107c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009930 tx=0x7ffb9402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/240690501 shutdown_connections 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/240690501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 0x7ffba410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/240690501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 0x7ffba4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/240690501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba4107c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/240690501 >> 172.21.15.62:0/240690501 conn(0x7ffba40fd5f0 msgr2=0x7ffba40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/240690501 shutdown_connections 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.750+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/240690501 wait complete. 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 Processor -- start 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 -- start start 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 0x7ffba419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 0x7ffba419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:20.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4111cd0 con 0x7ffba41078b0 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4111b50 con 0x7ffba41018b0 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffbaa9a9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4111e50 con 0x7ffba4102250 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59166/0 (socket says 172.21.15.62:59166) 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 -- 172.21.15.62:0/99167194 learned_addr learned my addr 172.21.15.62:0/99167194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba3fff640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 0x7ffba419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba37fe640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 0x7ffba419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 -- 172.21.15.62:0/99167194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 msgr2=0x7ffba419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 0x7ffba419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 -- 172.21.15.62:0/99167194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 msgr2=0x7ffba419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 0x7ffba419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.751+0000 7ffba8f46640 1 -- 172.21.15.62:0/99167194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba419f370 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffba8f46640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00b970 tx=0x7ffb9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c019070 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9c00cb40 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba419f660 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c007650 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.752+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba41a6f40 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.753+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba4101d10 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.754+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb9c01c070 con 0x7ffba41078b0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.755+0000 7ffba17fa640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 0x7ffb8c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.755+0000 7ffba3fff640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 0x7ffb8c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:20.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.755+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ffb9c0ce790 con 0x7ffba41078b0 2024-08-16T19:08:20.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.755+0000 7ffba3fff640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 0x7ffb8c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:20.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.759+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffb9c09ae60 con 0x7ffba41078b0 2024-08-16T19:08:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.973+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba41a7220 con 0x7ffba41078b0 2024-08-16T19:08:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.973+0000 7ffba17fa640 1 -- 172.21.15.62:0/99167194 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11854 (secure 0 0 0) 0x7ffb9c09f160 con 0x7ffba41078b0 2024-08-16T19:08:20.974 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:20.974 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"btime":"2024-08-16T19:08:14:618694+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:14.349923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24476},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24476,"qdb_cluster":[24476]},"id":1}]} 2024-08-16T19:08:20.975 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-08-16T19:08:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 msgr2=0x7ffb8c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 0x7ffb8c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb940058d0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 msgr2=0x7ffba419ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00b970 tx=0x7ffb9c00be30 comp rx=0 tx=0).stop 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 shutdown_connections 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba41078b0 0x7ffba419ec90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba4102250 0x7ffba419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffb8c0a2c00 0x7ffb8c0a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 --2- 172.21.15.62:0/99167194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffba41018b0 0x7ffba419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:20.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 >> 172.21.15.62:0/99167194 conn(0x7ffba40fd5f0 msgr2=0x7ffba40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:20.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 shutdown_connections 2024-08-16T19:08:20.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:20.976+0000 7ffbaa9a9640 1 -- 172.21.15.62:0/99167194 wait complete. 2024-08-16T19:08:21.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:20 smithi155 ceph-mon[37413]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:21.106 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:20 smithi161 ceph-mon[37576]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:21.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:21 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/99167194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:22.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:21 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/99167194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:22.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:21 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/99167194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- 172.21.15.62:0/2235823234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc103410 msgr2=0x7fe2dc1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2235823234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc103410 0x7fe2dc1078c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009930 tx=0x7fe2d002a420 comp rx=0 tx=0).stop 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- 172.21.15.62:0/2235823234 shutdown_connections 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2235823234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2235823234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc103410 0x7fe2dc1078c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2235823234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 0x7fe2dc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- 172.21.15.62:0/2235823234 >> 172.21.15.62:0/2235823234 conn(0x7fe2dc0fc500 msgr2=0x7fe2dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- 172.21.15.62:0/2235823234 shutdown_connections 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- 172.21.15.62:0/2235823234 wait complete. 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 Processor -- start 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.369+0000 7fe2e447c640 1 -- start start 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 0x7fe2dc1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 0x7fe2dc100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc111dd0 con 0x7fe2dc107fd0 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc111c50 con 0x7fe2dc102a60 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e447c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc111f50 con 0x7fe2dc103410 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38342/0 (socket says 172.21.15.62:38342) 2024-08-16T19:08:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 -- 172.21.15.62:0/2708794338 learned_addr learned my addr 172.21.15.62:0/2708794338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2218640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 0x7fe2dc1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e1a17640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 0x7fe2dc100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 -- 172.21.15.62:0/2708794338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 msgr2=0x7fe2dc100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 0x7fe2dc100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 -- 172.21.15.62:0/2708794338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 msgr2=0x7fe2dc1006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 0x7fe2dc1006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 -- 172.21.15.62:0/2708794338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a6830 con 0x7fe2dc107fd0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e1a17640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 0x7fe2dc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.370+0000 7fe2e2a19640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00eaa0 tx=0x7fe2cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.371+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00cdb0 con 0x7fe2dc107fd0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.371+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc005020 con 0x7fe2dc107fd0 2024-08-16T19:08:22.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.371+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a6b20 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.371+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc0116b0 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.371+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a7030 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.372+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc102e40 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.373+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe2cc005560 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.373+0000 7fe2db7fe640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 0x7fe2c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.374+0000 7fe2e2218640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 0x7fe2c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.374+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe2cc0cf460 con 0x7fe2dc107fd0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.374+0000 7fe2e2218640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 0x7fe2c40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40058c0 tx=0x7fe2d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:22.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.378+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe2cc016030 con 0x7fe2dc107fd0 2024-08-16T19:08:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.553+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2dc109fe0 con 0x7fe2dc107fd0 2024-08-16T19:08:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.555+0000 7fe2db7fe640 1 -- 172.21.15.62:0/2708794338 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24104 (secure 0 0 0) 0x7fe2cc09c180 con 0x7fe2dc107fd0 2024-08-16T19:08:22.556 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:22.556 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:14.349723+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 msgr2=0x7fe2c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 0x7fe2c40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40058c0 tx=0x7fe2d400a2d0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 msgr2=0x7fe2dc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00eaa0 tx=0x7fe2cc00ef60 comp rx=0 tx=0).stop 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 shutdown_connections 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2dc107fd0 0x7fe2dc101140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe2dc103410 0x7fe2dc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe2c40a2b80 0x7fe2c40a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 --2- 172.21.15.62:0/2708794338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2dc102a60 0x7fe2dc1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.557+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 >> 172.21.15.62:0/2708794338 conn(0x7fe2dc0fc500 msgr2=0x7fe2dc1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.558+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 shutdown_connections 2024-08-16T19:08:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:22.558+0000 7fe2e447c640 1 -- 172.21.15.62:0/2708794338 wait complete. 2024-08-16T19:08:22.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:22 smithi062 ceph-mon[32993]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:22.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:22 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2708794338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:23.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:22 smithi155 ceph-mon[37413]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:23.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:22 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2708794338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:23.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:22 smithi161 ceph-mon[37576]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:23.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:22 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2708794338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:23.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-16T19:08:05.356964+0000', 'modified': '2024-08-16T19:08:14.349923+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24476}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6839/2045382457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2045382457}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2045382457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24476, 'qdb_cluster': [24476]} 2024-08-16T19:08:23.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-16T19:08:23.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-16T19:08:23.457 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:23.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.918+0000 7f688a17e640 1 -- 172.21.15.62:0/971298913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6884102250 msgr2=0x7f688410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.918+0000 7f688a17e640 1 --2- 172.21.15.62:0/971298913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6884102250 0x7f688410e780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f687c009cd0 tx=0x7f687c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 -- 172.21.15.62:0/971298913 shutdown_connections 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 --2- 172.21.15.62:0/971298913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6884102250 0x7f688410e780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 --2- 172.21.15.62:0/971298913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68841018b0 0x7f6884101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 --2- 172.21.15.62:0/971298913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68841078b0 0x7f6884107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 -- 172.21.15.62:0/971298913 >> 172.21.15.62:0/971298913 conn(0x7f68840fd5f0 msgr2=0x7f68840ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 -- 172.21.15.62:0/971298913 shutdown_connections 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 -- 172.21.15.62:0/971298913 wait complete. 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 Processor -- start 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.919+0000 7f688a17e640 1 -- start start 2024-08-16T19:08:23.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6884102250 0x7f6884198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68841078b0 0x7f688419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111cd0 con 0x7f68841018b0 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111b50 con 0x7f6884102250 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111e50 con 0x7f68841078b0 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38360/0 (socket says 172.21.15.62:38360) 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 -- 172.21.15.62:0/654129215 learned_addr learned my addr 172.21.15.62:0/654129215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f687affd640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6884102250 0x7f6884198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 -- 172.21.15.62:0/654129215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68841078b0 msgr2=0x7f688419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68841078b0 0x7f688419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 -- 172.21.15.62:0/654129215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6884102250 msgr2=0x7f6884198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6884102250 0x7f6884198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 -- 172.21.15.62:0/654129215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688419d0e0 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68837fe640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f687400c920 tx=0x7f687400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68817fa640 1 -- 172.21.15.62:0/654129215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874007c70 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688419d3d0 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68817fa640 1 -- 172.21.15.62:0/654129215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6874007e10 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f68817fa640 1 -- 172.21.15.62:0/654129215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874018890 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.920+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68841a4cb0 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f68817fa640 1 -- 172.21.15.62:0/654129215 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6874018a70 con 0x7f68841018b0 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 Processor -- start 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 -- start start 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831e30 0x55a4988406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a498843010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a498796710 con 0x55a498831510 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a49872e770 con 0x55a498831e30 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f688c57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a4987926d0 con 0x55a498840c20 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f6883fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f6883fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38372/0 (socket says 172.21.15.62:38372) 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f6883fff640 1 -- 172.21.15.62:0/2194275683 learned_addr learned my addr 172.21.15.62:0/2194275683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.923+0000 7f68817fa640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 0x7f68700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 0x7f68700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f68837fe640 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831e30 0x55a4988406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f687affd640 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a498843010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f68817fa640 1 -- 172.21.15.62:0/654129215 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f68740cf480 con 0x7f68841018b0 2024-08-16T19:08:23.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 -- 172.21.15.62:0/2194275683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 msgr2=0x55a498843010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a498843010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 -- 172.21.15.62:0/2194275683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831e30 msgr2=0x55a4988406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831e30 0x55a4988406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 -- 172.21.15.62:0/2194275683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4987353e0 con 0x55a498831510 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f687c006010 tx=0x7f687c031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d3a17804e829a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6883fff640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 0x7f68700a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f687c009ca0 tx=0x7f687c048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6879ffb640 1 -- 172.21.15.62:0/2194275683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c050070 con 0x55a498831510 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6879ffb640 1 -- 172.21.15.62:0/2194275683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c036040 con 0x55a498831510 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.924+0000 7f6879ffb640 1 -- 172.21.15.62:0/2194275683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c030b40 con 0x55a498831510 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- 172.21.15.62:0/2194275683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 msgr2=0x55a4988318f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f687c006010 tx=0x7f687c031df0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- 172.21.15.62:0/2194275683 shutdown_connections 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a498843010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831e30 0x55a4988406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 --2- 172.21.15.62:0/2194275683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a4988318f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- 172.21.15.62:0/2194275683 >> 172.21.15.62:0/2194275683 conn(0x55a49882dbc0 msgr2=0x55a49882dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- 172.21.15.62:0/2194275683 shutdown_connections 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- 172.21.15.62:0/2194275683 wait complete. 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.925+0000 7f688c57c740 1 -- start start 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 Processor -- start 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 0x55a49883dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a49883e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a4988437f0 con 0x55a498831e30 2024-08-16T19:08:23.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a498751970 con 0x55a498840c20 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a498843970 con 0x55a498831510 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f688c57c740 10 client.0 _ll_register_callbacks cb 0x55a4984a57b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38378/0 (socket says 172.21.15.62:38378) 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 -- 172.21.15.62:0/1893840863 learned_addr learned my addr 172.21.15.62:0/1893840863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f6883fff640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a49883e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f68837fe640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 0x55a49883dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 -- 172.21.15.62:0/1893840863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 msgr2=0x55a49883dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 0x55a49883dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 -- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 msgr2=0x55a49883e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a49883e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 -- 172.21.15.62:0/1893840863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4985082b0 con 0x55a498831e30 2024-08-16T19:08:23.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f68837fe640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 0x55a49883dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f687affd640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f686c010500 tx=0x7f686c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6abf731cd37d99d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.926+0000 7f6862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01a020 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c036040 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f686c016750 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f686c025050 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 10 client.14700 fetch_fsmap learned FSMap version 16 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55a49884f540 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c036040 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f686c014b60 con 0x55a498831e30 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 10 client.14700 fetch_fsmap finished waiting for FSMap version 16 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 10 client.14700 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 20 client.14700 populate_metadata read hostname 'smithi062' 2024-08-16T19:08:23.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a498853050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 4 client.14700 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:08:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.927+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4987a3c80 con 0x55a498851be0 2024-08-16T19:08:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.928+0000 7f68837fe640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a498853050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.928+0000 7f68837fe640 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a498853050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c1404597c1d63c server_cookie=d2190d358d0399f6 in_seq=0 out_seq=0 2024-08-16T19:08:23.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.928+0000 7f6862ffd640 10 client.14700 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f6862ffd640 1 -- 172.21.15.62:0/1893840863 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4987a3c80 con 0x55a498851be0 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f6862ffd640 10 client.14700 handle_command_reply: tid=0 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 client.14700 shutdown 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 msgr2=0x55a49883e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f686c010500 tx=0x7f686c0109c0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 shutdown_connections 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a498853050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f6862ffd640 0 client.14700 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a49883e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49883e100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 --2- 172.21.15.62:0/1893840863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831510 0x55a49883dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 >> 172.21.15.62:0/1893840863 conn(0x55a49882dbc0 msgr2=0x55a498833920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 shutdown_connections 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.929+0000 7f688c57c740 1 -- 172.21.15.62:0/1893840863 wait complete. 2024-08-16T19:08:23.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.930+0000 7f688c57c740 20 client.14700 trim_cache size 0 max 16384 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 Processor -- start 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 -- start start 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 0x55a498853050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a49883e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a498851390 con 0x55a498831e30 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a498851a60 con 0x55a498831510 2024-08-16T19:08:23.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f688c57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a498852130 con 0x55a498840c20 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38394/0 (socket says 172.21.15.62:38394) 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 -- 172.21.15.62:0/2017781766 learned_addr learned my addr 172.21.15.62:0/2017781766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f6883fff640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a49883e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f68837fe640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 0x55a498853050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 -- 172.21.15.62:0/2017781766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 msgr2=0x55a49883e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a49883e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 -- 172.21.15.62:0/2017781766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 msgr2=0x55a498853050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 0x55a498853050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 -- 172.21.15.62:0/2017781766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a498829130 con 0x55a498831e30 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f6883fff640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a49883e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f68837fe640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 0x55a498853050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.948+0000 7f687affd640 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f686c000c00 tx=0x7f686c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2e3a1cf91a625c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f68797fa640 1 -- 172.21.15.62:0/2017781766 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c02e070 con 0x55a498831e30 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f68797fa640 1 -- 172.21.15.62:0/2017781766 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c00bd20 con 0x55a498831e30 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f68797fa640 1 -- 172.21.15.62:0/2017781766 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01a040 con 0x55a498831e30 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 -- 172.21.15.62:0/2017781766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 msgr2=0x55a49884c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f686c000c00 tx=0x7f686c00b480 comp rx=0 tx=0).stop 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 -- 172.21.15.62:0/2017781766 shutdown_connections 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498840c20 0x55a49883e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831e30 0x55a49884c310 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 --2- 172.21.15.62:0/2017781766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498831510 0x55a498853050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 -- 172.21.15.62:0/2017781766 >> 172.21.15.62:0/2017781766 conn(0x55a49882dbc0 msgr2=0x55a49884bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 -- 172.21.15.62:0/2017781766 shutdown_connections 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.949+0000 7f688c57c740 1 -- 172.21.15.62:0/2017781766 wait complete. 2024-08-16T19:08:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 -- start start 2024-08-16T19:08:23.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 Processor -- start 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 0x55a498854de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a4988553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a498852ed0 con 0x55a498831510 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55a498852800 con 0x55a498840c20 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a4988535a0 con 0x55a498831e30 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f688c57c740 10 client.0 _ll_register_callbacks cb 0x7f686c00b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f68837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f68837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38400/0 (socket says 172.21.15.62:38400) 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.950+0000 7f68837fe640 1 -- 172.21.15.62:0/388607616 learned_addr learned my addr 172.21.15.62:0/388607616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f687affd640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 0x55a498854de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6883fff640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a4988553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 -- 172.21.15.62:0/388607616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 msgr2=0x55a498854de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 0x55a498854de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 -- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 msgr2=0x55a4988553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a4988553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 -- 172.21.15.62:0/388607616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a498855bd0 con 0x55a498831510 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f687affd640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 0x55a498854de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f68837fe640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6874097f00 tx=0x7f687401dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52a045dde642431 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68740b4030 con 0x55a498831510 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f68740663f0 con 0x55a498831510 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c00bd20 con 0x55a498831510 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f68740b9050 con 0x55a498831510 2024-08-16T19:08:23.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f688c57c740 10 client.14706 fetch_fsmap learned FSMap version 16 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x55a4988bc410 con 0x55a498831510 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.951+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c00bd20 con 0x55a498831510 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f6874098110 con 0x55a498831510 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 10 client.14706 fetch_fsmap finished waiting for FSMap version 16 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 10 client.14706 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 20 client.14706 populate_metadata read hostname 'smithi062' 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a4988bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 4 client.14706 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "status"}) -- 0x55a49879e190 con 0x55a498851be0 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f687affd640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a4988bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f6862ffd640 10 client.14706 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f687affd640 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a4988bfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14905ec4523a490c server_cookie=2fb14a090222b5d2 in_seq=0 out_seq=0 2024-08-16T19:08:23.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f6862ffd640 1 -- 172.21.15.62:0/388607616 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a49879e190 con 0x55a498851be0 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.952+0000 7f6862ffd640 10 client.14706 handle_command_reply: tid=0 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 client.14706 shutdown 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 msgr2=0x55a49885bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6874097f00 tx=0x7f687401dcd0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 shutdown_connections 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55a498851be0 0x55a4988bfb30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a498840c20 0x55a4988553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a498831e30 0x55a498854de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 --2- 172.21.15.62:0/388607616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a498831510 0x55a49885bab0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 >> 172.21.15.62:0/388607616 conn(0x55a49882dbc0 msgr2=0x55a49884c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f6862ffd640 0 client.14706 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 shutdown_connections 2024-08-16T19:08:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 1 -- 172.21.15.62:0/388607616 wait complete. 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688c57c740 20 client.14706 trim_cache size 0 max 16384 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24476, 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 18.375430187999999, 2024-08-16T19:08:23.960 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 16, 2024-08-16T19:08:23.961 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 88, 2024-08-16T19:08:23.961 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:08:23.961 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 29.413612268000001 2024-08-16T19:08:23.961 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:08:23.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 msgr2=0x7f68700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 0x7f68700a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f687c009ca0 tx=0x7f687c048000 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 msgr2=0x7f6884198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.953+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f687400c920 tx=0x7f687400cde0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 shutdown_connections 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68841078b0 0x7f688419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f68700a2b80 0x7f68700a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6884102250 0x7f6884198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 --2- 172.21.15.62:0/654129215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68841018b0 0x7f6884198130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 >> 172.21.15.62:0/654129215 conn(0x7f68840fd5f0 msgr2=0x7f68840fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 shutdown_connections 2024-08-16T19:08:23.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:23.954+0000 7f688a17e640 1 -- 172.21.15.62:0/654129215 wait complete. 2024-08-16T19:08:24.669 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-16T19:08:24.669 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:24.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-16T19:08:24.936 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:24.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:24 smithi062 ceph-mon[32993]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:25.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:24 smithi155 ceph-mon[37413]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:25.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:24 smithi161 ceph-mon[37576]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:25.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 -- 172.21.15.62:0/106096061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 msgr2=0x7fe79810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 --2- 172.21.15.62:0/106096061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79810e980 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe788009cd0 tx=0x7fe78802a910 comp rx=0 tx=0).stop 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 -- 172.21.15.62:0/106096061 shutdown_connections 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 --2- 172.21.15.62:0/106096061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79810e980 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 --2- 172.21.15.62:0/106096061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.394+0000 7fe79faaa640 1 --2- 172.21.15.62:0/106096061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 0x7fe798102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- 172.21.15.62:0/106096061 >> 172.21.15.62:0/106096061 conn(0x7fe7980fc500 msgr2=0x7fe7980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- 172.21.15.62:0/106096061 shutdown_connections 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- 172.21.15.62:0/106096061 wait complete. 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 Processor -- start 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- start start 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 0x7fe7981982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:25.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe798111e70 con 0x7fe798108010 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe798111cf0 con 0x7fe798103490 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.395+0000 7fe79faaa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe798111ff0 con 0x7fe798102ae0 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38406/0 (socket says 172.21.15.62:38406) 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79d846640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 0x7fe7981982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 -- 172.21.15.62:0/3816195250 learned_addr learned my addr 172.21.15.62:0/3816195250 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79d045640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 -- 172.21.15.62:0/3816195250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 msgr2=0x7fe7981982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 0x7fe7981982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 -- 172.21.15.62:0/3816195250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 msgr2=0x7fe7981987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:25.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 -- 172.21.15.62:0/3816195250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79819d250 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79d045640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79e047640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe788005e30 tx=0x7fe788003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788031720 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7880318c0 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79819d4e0 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78802fb80 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.396+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a4e20 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.397+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe798102ec0 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.399+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe78802fd20 con 0x7fe798108010 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.400+0000 7fe796ffd640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 0x7fe7800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.400+0000 7fe79d846640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 0x7fe7800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:25.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.400+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe7880f0740 con 0x7fe798108010 2024-08-16T19:08:25.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.400+0000 7fe79d846640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 0x7fe7800a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe790005d00 tx=0x7fe79000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:25.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.404+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7880bd460 con 0x7fe798108010 2024-08-16T19:08:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.617+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe7981a4ff0 con 0x7fe798108010 2024-08-16T19:08:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.618+0000 7fe796ffd640 1 -- 172.21.15.62:0/3816195250 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7fe7880bcd60 con 0x7fe798108010 2024-08-16T19:08:25.618 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 msgr2=0x7fe7800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 0x7fe7800a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe790005d00 tx=0x7fe79000b410 comp rx=0 tx=0).stop 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 msgr2=0x7fe79819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe788005e30 tx=0x7fe788003f10 comp rx=0 tx=0).stop 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 shutdown_connections 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe798108010 0x7fe79819cb70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe7800a2c00 0x7fe7800a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe798103490 0x7fe7981987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 --2- 172.21.15.62:0/3816195250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe798102ae0 0x7fe7981982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 >> 172.21.15.62:0/3816195250 conn(0x7fe7980fc500 msgr2=0x7fe7980fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 shutdown_connections 2024-08-16T19:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:25.620+0000 7fe79faaa640 1 -- 172.21.15.62:0/3816195250 wait complete. 2024-08-16T19:08:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3816195250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:08:26.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:25 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3816195250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:08:26.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:25 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3816195250' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:08:26.188 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-16T19:08:26.189 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-16T19:08:26.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:26.189 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-08-16T19:08:26.206 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:08:26.207 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42293sec preferred_lft 42293sec 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-08-16T19:08:26.208 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:08:26.208 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-16T19:08:26.209 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:08:26.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:08:26.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:26.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:26.418 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-16T19:08:26.445 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-08-16T19:08:26.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:26.447 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-08-16T19:08:26.519 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-08-16T19:08:26.520 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-16T19:08:26.520 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-16T19:08:26.520 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-16T19:08:26.520 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-16T19:08:26.520 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:08:26.520 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:08:26.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-16T19:08:26.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-16T19:08:26.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-16T19:08:26.521 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:08:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:08:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:26.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:26.813 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-08-16T19:08:26.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:26.829 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-08-16T19:08:26.885 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:08:26.886 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:08:26.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:08:26.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-16T19:08:26.886 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:08:26.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:08:26.984 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:26 smithi062 ceph-mon[32993]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:27.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:27.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-16T19:08:27.007 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:08:27.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:08:27.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:26 smithi155 ceph-mon[37413]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:27.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:27.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:26 smithi161 ceph-mon[37576]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:27.141 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:08:27.141 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:08:27.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-08-16T19:08:27.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-08-16T19:08:27.141 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:08:27.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:08:27.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:27.222 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-16T19:08:27.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-16T19:08:27.222 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:27.237 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-16T19:08:27.238 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-16T19:08:27.238 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:27.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-08-16T19:08:27.361 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-08-16T19:08:27.418 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-08-16T19:08:27.419 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/0f4ea43e58167066226e086fb0ecde28c82a24e9cffc7a72540def892b2c1a3b/merged 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/4984cadcd0397cb9eb356ee51a3ec63b2291c75f17a7e2a31e8c35e198b4d786/merged 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/0 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/25896d1c2f605735d55cb5d151f3ede4c7e89d1ea506339e9a9c53e7f58ab263/merged 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/7adbf0706ab679ae67494b504b8be2dbff92f690629486f4488be8b5362ca155/merged 2024-08-16T19:08:27.420 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/f388ab2177b589a9e1ba0c66cdaffa969e085d98b4f90809b2abf232f371c565/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/b7f05a1103f56622cdb64fb23c641d84ad6e46ac91f976cff70ad96725d09290/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/02dc5b7a88ed3891be216d6191ad8780e3c0b7cc852e87bd688c17736944ee43/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/68f2f6342a4266a9ec7476f302a99a9d7d300f2b8c54aecbda541b98bbc31f0d/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/d258cbf5c7454a011097126f39d4ba6d2e362808b27d03ef1da74154d0160716/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/cc8c6d664911f5d6dfd4dbb46bbcfb01b4a4fb972d44fe96d09a31fa362c09eb/merged 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:08:27.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:27.421 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-08-16T19:08:27.474 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-16T19:08:27.474 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-16T19:08:27.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo modprobe fuse 2024-08-16T19:08:27.541 DEBUG:teuthology.orchestra.run.smithi062:> cat /proc/self/mounts | awk '{print $2}' 2024-08-16T19:08:27.598 INFO:teuthology.orchestra.run.smithi062.stdout:/proc 2024-08-16T19:08:27.598 INFO:teuthology.orchestra.run.smithi062.stdout:/sys 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/dev 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/security 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/shm 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/pts 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/run 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/cgroup 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/pstore 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/bpf 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/ 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/selinux 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/hugepages 2024-08-16T19:08:27.599 INFO:teuthology.orchestra.run.smithi062.stdout:/dev/mqueue 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/debug 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/tracing 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/fs/fuse/connections 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-sysctl.service 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/sys/kernel/config 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/nfs/rpc_pipefs 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/1000 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/ceph 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/0f4ea43e58167066226e086fb0ecde28c82a24e9cffc7a72540def892b2c1a3b/merged 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/4984cadcd0397cb9eb356ee51a3ec63b2291c75f17a7e2a31e8c35e198b4d786/merged 2024-08-16T19:08:27.600 INFO:teuthology.orchestra.run.smithi062.stdout:/run/user/0 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/proc/sys/fs/binfmt_misc 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/25896d1c2f605735d55cb5d151f3ede4c7e89d1ea506339e9a9c53e7f58ab263/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/7adbf0706ab679ae67494b504b8be2dbff92f690629486f4488be8b5362ca155/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/f388ab2177b589a9e1ba0c66cdaffa969e085d98b4f90809b2abf232f371c565/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/b7f05a1103f56622cdb64fb23c641d84ad6e46ac91f976cff70ad96725d09290/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/02dc5b7a88ed3891be216d6191ad8780e3c0b7cc852e87bd688c17736944ee43/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/68f2f6342a4266a9ec7476f302a99a9d7d300f2b8c54aecbda541b98bbc31f0d/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/d258cbf5c7454a011097126f39d4ba6d2e362808b27d03ef1da74154d0160716/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/var/lib/containers/storage/overlay/cc8c6d664911f5d6dfd4dbb46bbcfb01b4a4fb972d44fe96d09a31fa362c09eb/merged 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:08:27.601 INFO:teuthology.orchestra.run.smithi062.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:08:27.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:27.602 DEBUG:teuthology.orchestra.run.smithi062:> ls /sys/fs/fuse/connections 2024-08-16T19:08:27.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:2024-08-16T19:08:27.624+0000 7f7f6db89480 -1 init, newargv = 0x5579d0f86960 newargc=15 2024-08-16T19:08:27.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[76955]: starting ceph client 2024-08-16T19:08:27.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[76955]: starting fuse 2024-08-16T19:08:27.641 INFO:teuthology.orchestra.run.smithi062.stdout:119 2024-08-16T19:08:27.642 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo stdin-killer -- python3 -c ' 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> import glob 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> import re 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> import subprocess 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> def _find_admin_socket(client_name): 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-16T19:08:27.642 DEBUG:teuthology.orchestra.run.smithi062:> files = glob.glob(asok_path) 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> # Given a non-glob path, it better be there 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> if "*" not in asok_path: 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> assert(len(files) == 1) 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> return files[0] 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> for f in files: 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> contents = proc_f.read() 2024-08-16T19:08:27.643 DEBUG:teuthology.orchestra.run.smithi062:> if mountpoint in contents: 2024-08-16T19:08:27.644 DEBUG:teuthology.orchestra.run.smithi062:> return f 2024-08-16T19:08:27.644 DEBUG:teuthology.orchestra.run.smithi062:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-16T19:08:27.644 DEBUG:teuthology.orchestra.run.smithi062:> 2024-08-16T19:08:27.644 DEBUG:teuthology.orchestra.run.smithi062:> print(_find_admin_socket("client.admin")) 2024-08-16T19:08:27.644 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-08-16T19:08:27.764 INFO:teuthology.orchestra.run.smithi062.stdout:/var/run/ceph/ceph-client.admin.76955.asok 2024-08-16T19:08:27.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:27.775 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76955.asok 2024-08-16T19:08:27.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:27.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76955.asok status 2024-08-16T19:08:27.914 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:08:27.914 INFO:teuthology.orchestra.run.smithi062.stdout: "metadata": { 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_sha1": "d863d21c9d1db9c3453c09dacd68766969720a44", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.1.0-1363-gd863d21c (d863d21c9d1db9c3453c09dacd68766969720a44) squid (rc)", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "entity_id": "admin", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": "smithi062", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "pid": "76955", 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "root": "/" 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_count": 0, 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "dentry_pinned_count": 0, 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 14718, 2024-08-16T19:08:27.915 INFO:teuthology.orchestra.run.smithi062.stdout: "inst": { 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": { 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "client", 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "num": 14718 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2742345162 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": { 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "192.168.0.1:0", 2024-08-16T19:08:27.916 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2742345162 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "inst_str": "client.14718 192.168.0.1:0/2742345162", 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "addr_str": "192.168.0.1:0/2742345162", 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "inode_count": 1, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_epoch": 16, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch": 88, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_epoch_barrier": 0, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "blocklisted": false, 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs" 2024-08-16T19:08:27.917 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:08:27.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:27.924 DEBUG:teuthology.orchestra.run.smithi062:> stat --file-system '--printf=%T 2024-08-16T19:08:27.924 DEBUG:teuthology.orchestra.run.smithi062:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:27.940 INFO:teuthology.orchestra.run.smithi062.stdout:fuseblk 2024-08-16T19:08:27.940 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:27.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:27.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:28.007 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-16T19:08:28.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-16T19:08:28.118 INFO:teuthology.orchestra.run.smithi062.stdout:# file: . 2024-08-16T19:08:28.118 INFO:teuthology.orchestra.run.smithi062.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-16T19:08:28.118 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:28.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:08:28.124 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:08:28.124 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-16T19:08:28.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-16T19:08:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:28.901 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:28 smithi062 ceph-mon[32993]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 -- 172.21.15.62:0/2492281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e41022a0 msgr2=0x7f31e41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/2492281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e41022a0 0x7f31e41026e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009900 tx=0x7f31d802a480 comp rx=0 tx=0).stop 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 -- 172.21.15.62:0/2492281 shutdown_connections 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/2492281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e4102c20 0x7f31e410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/2492281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e41022a0 0x7f31e41026e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:28.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/2492281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 0x7f31e4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 -- 172.21.15.62:0/2492281 >> 172.21.15.62:0/2492281 conn(0x7f31e40fc500 msgr2=0x7f31e40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 -- 172.21.15.62:0/2492281 shutdown_connections 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.900+0000 7f31ec7e6640 1 -- 172.21.15.62:0/2492281 wait complete. 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 Processor -- start 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 -- start start 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 0x7f31e419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e41022a0 0x7f31e419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110cb0 con 0x7f31e4102c20 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110b30 con 0x7f31e4100b50 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ec7e6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4110e30 con 0x7f31e41022a0 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ead83640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ead83640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38420/0 (socket says 172.21.15.62:38420) 2024-08-16T19:08:28.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ead83640 1 -- 172.21.15.62:0/594412567 learned_addr learned my addr 172.21.15.62:0/594412567 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31e9d81640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e41022a0 0x7f31e419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.901+0000 7f31ea582640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 0x7f31e419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 -- 172.21.15.62:0/594412567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e41022a0 msgr2=0x7f31e419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e41022a0 0x7f31e419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 -- 172.21.15.62:0/594412567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 msgr2=0x7f31e419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 0x7f31e419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 -- 172.21.15.62:0/594412567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e419f3b0 con 0x7f31e4102c20 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ead83640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31d400eaa0 tx=0x7f31d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d400cdb0 con 0x7f31e4102c20 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d4005020 con 0x7f31e4102c20 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e419f6a0 con 0x7f31e4102c20 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40116b0 con 0x7f31e4102c20 2024-08-16T19:08:28.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.902+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a6f80 con 0x7f31e4102c20 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.903+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4100f30 con 0x7f31e4102c20 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.904+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f31d4005560 con 0x7f31e4102c20 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.905+0000 7f31e37fe640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.905+0000 7f31ea582640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.905+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f31d40cf420 con 0x7f31e4102c20 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.906+0000 7f31ea582640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 0x7f31cc0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bbe0 tx=0x7f31dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:28.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:28.910+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f31d4016030 con 0x7f31e4102c20 2024-08-16T19:08:29.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:28 smithi155 ceph-mon[37413]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.099+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f31e41a71e0 con 0x7f31e4102c20 2024-08-16T19:08:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.100+0000 7f31e37fe640 1 -- 172.21.15.62:0/594412567 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f31d409baf0 con 0x7f31e4102c20 2024-08-16T19:08:29.100 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:29.100 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3183618050 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:29.100 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1123206672 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4100589739 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2033319205 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3894293084 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3941316438 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 msgr2=0x7f31cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 0x7f31cc0a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bbe0 tx=0x7f31dc00b390 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 msgr2=0x7f31e419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f31d400eaa0 tx=0x7f31d400ef60 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 shutdown_connections 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31e4102c20 0x7f31e419ecd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31e41022a0 0x7f31e419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f31cc0a2b80 0x7f31cc0a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 --2- 172.21.15.62:0/594412567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31e4100b50 0x7f31e419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 >> 172.21.15.62:0/594412567 conn(0x7f31e40fc500 msgr2=0x7f31e4069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:29.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 shutdown_connections 2024-08-16T19:08:29.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:29.102+0000 7f31ec7e6640 1 -- 172.21.15.62:0/594412567 wait complete. 2024-08-16T19:08:29.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:28 smithi161 ceph-mon[37576]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:29.739 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:29 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/594412567' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:29.766 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:08:29.767 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-16T19:08:29.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-16T19:08:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:29 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/594412567' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:30.086 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:30.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:29 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/594412567' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.575+0000 7fd6da894640 1 -- 172.21.15.62:0/2462759180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41018b0 msgr2=0x7fd6d4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.575+0000 7fd6da894640 1 --2- 172.21.15.62:0/2462759180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41018b0 0x7fd6d4101d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009930 tx=0x7fd6c802a420 comp rx=0 tx=0).stop 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.575+0000 7fd6da894640 1 -- 172.21.15.62:0/2462759180 shutdown_connections 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 --2- 172.21.15.62:0/2462759180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d4102250 0x7fd6d410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 --2- 172.21.15.62:0/2462759180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41018b0 0x7fd6d4101d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 --2- 172.21.15.62:0/2462759180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d41078b0 0x7fd6d4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 -- 172.21.15.62:0/2462759180 >> 172.21.15.62:0/2462759180 conn(0x7fd6d40fd5f0 msgr2=0x7fd6d40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 -- 172.21.15.62:0/2462759180 shutdown_connections 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 -- 172.21.15.62:0/2462759180 wait complete. 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 Processor -- start 2024-08-16T19:08:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.576+0000 7fd6da894640 1 -- start start 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d41018b0 0x7fd6d419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 0x7fd6d419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111cf0 con 0x7fd6d41078b0 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111b70 con 0x7fd6d41018b0 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6da894640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4111e70 con 0x7fd6d4102250 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38436/0 (socket says 172.21.15.62:38436) 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 -- 172.21.15.62:0/341544171 learned_addr learned my addr 172.21.15.62:0/341544171 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d37fe640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 0x7fd6d419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:30.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d3fff640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d41018b0 0x7fd6d419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 -- 172.21.15.62:0/341544171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 msgr2=0x7fd6d419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 0x7fd6d419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 -- 172.21.15.62:0/341544171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d41018b0 msgr2=0x7fd6d419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d41018b0 0x7fd6d419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 -- 172.21.15.62:0/341544171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d419f400 con 0x7fd6d41078b0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d37fe640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 0x7fd6d419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.577+0000 7fd6d8e31640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b970 tx=0x7fd6cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.578+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc019070 con 0x7fd6d41078b0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.578+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d419f6f0 con 0x7fd6d41078b0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.578+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6cc00cb40 con 0x7fd6d41078b0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.578+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc007650 con 0x7fd6d41078b0 2024-08-16T19:08:30.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.578+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d41a6fd0 con 0x7fd6d41078b0 2024-08-16T19:08:30.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.579+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d4101d10 con 0x7fd6d41078b0 2024-08-16T19:08:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.580+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd6cc01c070 con 0x7fd6d41078b0 2024-08-16T19:08:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.580+0000 7fd6d17fa640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 0x7fd6bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.581+0000 7fd6d3fff640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 0x7fd6bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.581+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fd6cc0ce920 con 0x7fd6d41078b0 2024-08-16T19:08:30.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.581+0000 7fd6d3fff640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 0x7fd6bc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005fd0 tx=0x7fd6c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:30.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.585+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd6cc09b640 con 0x7fd6d41078b0 2024-08-16T19:08:30.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:30 smithi062 ceph-mon[32993]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:30.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.776+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6d41a7230 con 0x7fd6d41078b0 2024-08-16T19:08:30.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.777+0000 7fd6d17fa640 1 -- 172.21.15.62:0/341544171 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7fd6cc09af40 con 0x7fd6d41078b0 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3183618050 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1123206672 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4100589739 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2033319205 2024-08-17T19:01:57.573528+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3894293084 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:30.780 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3941316438 2024-08-17T19:01:33.379605+0000 2024-08-16T19:08:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:listed 10 entries 2024-08-16T19:08:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 msgr2=0x7fd6bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 0x7fd6bc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005fd0 tx=0x7fd6c4005850 comp rx=0 tx=0).stop 2024-08-16T19:08:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 msgr2=0x7fd6d419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b970 tx=0x7fd6cc00be30 comp rx=0 tx=0).stop 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 shutdown_connections 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6d41078b0 0x7fd6d419ed20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d4102250 0x7fd6d419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd6bc0a2b80 0x7fd6bc0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 --2- 172.21.15.62:0/341544171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d41018b0 0x7fd6d419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 >> 172.21.15.62:0/341544171 conn(0x7fd6d40fd5f0 msgr2=0x7fd6d40fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 shutdown_connections 2024-08-16T19:08:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:30.779+0000 7fd6da894640 1 -- 172.21.15.62:0/341544171 wait complete. 2024-08-16T19:08:31.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:30 smithi155 ceph-mon[37413]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:31.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:30 smithi161 ceph-mon[37576]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:31.387 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi062.front.sepia.ceph.com... 2024-08-16T19:08:31.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:31.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-16T19:08:31.416 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi062.stderr:ceph-fuse[76955]: fuse finished with error 0 and tester_r 0 2024-08-16T19:08:31.417 INFO:teuthology.orchestra.run:waiting for 300 2024-08-16T19:08:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:31 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/341544171' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:32.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:31 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/341544171' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:32.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:31 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/341544171' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:08:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:32 smithi062 ceph-mon[32993]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:33.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:32 smithi155 ceph-mon[37413]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:33.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:32 smithi161 ceph-mon[37576]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:08:34.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:34 smithi062 ceph-mon[32993]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:35.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:34 smithi155 ceph-mon[37413]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:34 smithi161 ceph-mon[37576]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:36.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:36 smithi062 ceph-mon[32993]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:37.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:36 smithi155 ceph-mon[37413]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:37.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:36 smithi161 ceph-mon[37576]: pgmap v277: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:37.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:08:37.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:08:37.519 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-16T19:08:37.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:37.860 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 -- 172.21.15.62:0/987083202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa4101150 msgr2=0x7faaa410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 --2- 172.21.15.62:0/987083202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa4101150 0x7faaa410d680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009cd0 tx=0x7faa9c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 -- 172.21.15.62:0/987083202 shutdown_connections 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 --2- 172.21.15.62:0/987083202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa4101150 0x7faaa410d680 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 --2- 172.21.15.62:0/987083202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa41007b0 0x7faaa4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 --2- 172.21.15.62:0/987083202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41067b0 0x7faaa4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 -- 172.21.15.62:0/987083202 >> 172.21.15.62:0/987083202 conn(0x7faaa40fc4d0 msgr2=0x7faaa40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 -- 172.21.15.62:0/987083202 shutdown_connections 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 -- 172.21.15.62:0/987083202 wait complete. 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.326+0000 7faaaa366640 1 Processor -- start 2024-08-16T19:08:38.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 -- start start 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 0x7faaa4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa4101150 0x7faaa4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4110bb0 con 0x7faaa41067b0 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4110a30 con 0x7faaa41007b0 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaaa366640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4110d30 con 0x7faaa4101150 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34090/0 (socket says 172.21.15.62:34090) 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 -- 172.21.15.62:0/3369093730 learned_addr learned my addr 172.21.15.62:0/3369093730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faa9bfff640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa4101150 0x7faaa4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa3fff640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 0x7faaa4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:38.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 -- 172.21.15.62:0/3369093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa4101150 msgr2=0x7faaa4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa4101150 0x7faaa4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 -- 172.21.15.62:0/3369093730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 msgr2=0x7faaa4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 0x7faaa4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa8903640 1 -- 172.21.15.62:0/3369093730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa41a4320 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.327+0000 7faaa3fff640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 0x7faaa4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaa8903640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c004450 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9c003e10 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa41a45b0 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c046480 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.328+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa41a4b40 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.329+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa4100c10 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.330+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa9c0045f0 con 0x7faaa41067b0 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.331+0000 7faaa1ffb640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 0x7faa900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:38.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.331+0000 7faaa3fff640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 0x7faa900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:38.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.331+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7faa9c02e030 con 0x7faaa41067b0 2024-08-16T19:08:38.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.331+0000 7faaa3fff640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 0x7faa900a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faa94005e10 tx=0x7faa94005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:38.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.335+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faa9c04b060 con 0x7faaa41067b0 2024-08-16T19:08:38.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.511+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaa41a4e60 con 0x7faaa41067b0 2024-08-16T19:08:38.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.512+0000 7faaa1ffb640 1 -- 172.21.15.62:0/3369093730 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24104 (secure 0 0 0) 0x7faa9c039380 con 0x7faaa41067b0 2024-08-16T19:08:38.514 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:38.514 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:14.349723+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.514+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 msgr2=0x7faa900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.514+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 0x7faa900a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faa94005e10 tx=0x7faa94005da0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.514+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 msgr2=0x7faaa41999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.514+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa9c006010 tx=0x7faa9c003c20 comp rx=0 tx=0).stop 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 shutdown_connections 2024-08-16T19:08:38.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaa41067b0 0x7faaa41999e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa4101150 0x7faaa4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7faa900a2c00 0x7faa900a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 --2- 172.21.15.62:0/3369093730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faaa41007b0 0x7faaa4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 >> 172.21.15.62:0/3369093730 conn(0x7faaa40fc4d0 msgr2=0x7faaa40fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 shutdown_connections 2024-08-16T19:08:38.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:38.515+0000 7faaaa366640 1 -- 172.21.15.62:0/3369093730 wait complete. 2024-08-16T19:08:38.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:38 smithi062 ceph-mon[32993]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:38.832 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:38 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3369093730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:38 smithi155 ceph-mon[37413]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:38 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3369093730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:38 smithi161 ceph-mon[37576]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:38 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3369093730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:39.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:39.436 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:39.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.875+0000 7f29b4bf2640 1 -- 172.21.15.62:0/231173028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010bea0 msgr2=0x7f29b010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.875+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/231173028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010bea0 0x7f29b010e340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009ca0 tx=0x7f29a802a950 comp rx=0 tx=0).stop 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 -- 172.21.15.62:0/231173028 shutdown_connections 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/231173028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b010bea0 0x7f29b010e340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/231173028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b0103390 0x7f29b010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/231173028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 0x7f29b0102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 -- 172.21.15.62:0/231173028 >> 172.21.15.62:0/231173028 conn(0x7f29b00fc4d0 msgr2=0x7f29b00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 -- 172.21.15.62:0/231173028 shutdown_connections 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 -- 172.21.15.62:0/231173028 wait complete. 2024-08-16T19:08:39.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.876+0000 7f29b4bf2640 1 Processor -- start 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 -- start start 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 0x7f29b01006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 0x7f29b0101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0111c60 con 0x7f29b0103390 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0111ae0 con 0x7f29b0102a70 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29b4bf2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0111de0 con 0x7f29b010bea0 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34110/0 (socket says 172.21.15.62:34110) 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 -- 172.21.15.62:0/633856897 learned_addr learned my addr 172.21.15.62:0/633856897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:39.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29ae59c640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 0x7f29b01006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29aed9d640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 0x7f29b0101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 -- 172.21.15.62:0/633856897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 msgr2=0x7f29b0101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 0x7f29b0101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 -- 172.21.15.62:0/633856897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 msgr2=0x7f29b01006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.877+0000 7f29add9b640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 0x7f29b01006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f29add9b640 1 -- 172.21.15.62:0/633856897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01aaa30 con 0x7f29b0103390 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f29aed9d640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 0x7f29b0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f29add9b640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29a400d950 tx=0x7f29a400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4017030 con 0x7f29b0103390 2024-08-16T19:08:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01aac60 con 0x7f29b0103390 2024-08-16T19:08:39.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a400be10 con 0x7f29b0103390 2024-08-16T19:08:39.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a40055f0 con 0x7f29b0103390 2024-08-16T19:08:39.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.878+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01ab1a0 con 0x7f29b0103390 2024-08-16T19:08:39.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.879+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b006ada0 con 0x7f29b0103390 2024-08-16T19:08:39.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.881+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29a4009a30 con 0x7f29b0103390 2024-08-16T19:08:39.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.882+0000 7f299f7fe640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 0x7f29980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:39.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.882+0000 7f29ae59c640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 0x7f29980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:39.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.882+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f29a40cf140 con 0x7f29b0103390 2024-08-16T19:08:39.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.882+0000 7f29ae59c640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 0x7f29980a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f29a000abe0 tx=0x7f29a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:39.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:39.885+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f29a409bee0 con 0x7f29b0103390 2024-08-16T19:08:40.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.062+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b01ab530 con 0x7f29b0103390 2024-08-16T19:08:40.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.063+0000 7f299f7fe640 1 -- 172.21.15.62:0/633856897 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24104 (secure 0 0 0) 0x7f29a409b7e0 con 0x7f29b0103390 2024-08-16T19:08:40.065 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:40.066 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":88,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:14.349723+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:08:40.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.065+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 msgr2=0x7f29980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:40.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.065+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 0x7f29980a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f29a000abe0 tx=0x7f29a000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:08:40.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.065+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 msgr2=0x7f29b0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:40.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.065+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29a400d950 tx=0x7f29a400dd50 comp rx=0 tx=0).stop 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 shutdown_connections 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29b010bea0 0x7f29b0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29b0103390 0x7f29b0100c10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29980a2c00 0x7f29980a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 --2- 172.21.15.62:0/633856897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29b0102a70 0x7f29b01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 >> 172.21.15.62:0/633856897 conn(0x7f29b00fc4d0 msgr2=0x7f29b00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 shutdown_connections 2024-08-16T19:08:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:40.066+0000 7f29b4bf2640 1 -- 172.21.15.62:0/633856897 wait complete. 2024-08-16T19:08:40.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-16T19:08:40.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:40 smithi062 ceph-mon[32993]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:40.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:40 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/633856897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:41.022 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:40 smithi155 ceph-mon[37413]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:40 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/633856897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:40 smithi161 ceph-mon[37576]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:40 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/633856897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:08:41.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 -- 172.21.15.62:0/719792887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 msgr2=0x7fbab0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 --2- 172.21.15.62:0/719792887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab0101d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009930 tx=0x7fbaa402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 -- 172.21.15.62:0/719792887 shutdown_connections 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 --2- 172.21.15.62:0/719792887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab0102250 0x7fbab010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 --2- 172.21.15.62:0/719792887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab0101d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 --2- 172.21.15.62:0/719792887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab01078b0 0x7fbab0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 -- 172.21.15.62:0/719792887 >> 172.21.15.62:0/719792887 conn(0x7fbab00fd5f0 msgr2=0x7fbab00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 -- 172.21.15.62:0/719792887 shutdown_connections 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.478+0000 7fbab68f8640 1 -- 172.21.15.62:0/719792887 wait complete. 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 Processor -- start 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 -- start start 2024-08-16T19:08:41.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 0x7fbab0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 0x7fbab019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0111cd0 con 0x7fbab01018b0 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0111b50 con 0x7fbab01078b0 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbab68f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0111e50 con 0x7fbab0102250 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47712/0 (socket says 172.21.15.62:47712) 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.479+0000 7fbaaffff640 1 -- 172.21.15.62:0/1539431464 learned_addr learned my addr 172.21.15.62:0/1539431464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaf7fe640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 0x7fbab0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbab4e95640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 0x7fbab019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 -- 172.21.15.62:0/1539431464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 msgr2=0x7fbab0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 0x7fbab0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 -- 172.21.15.62:0/1539431464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 msgr2=0x7fbab019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 0x7fbab019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 -- 172.21.15.62:0/1539431464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab019d180 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaf7fe640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 0x7fbab0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbab4e95640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 0x7fbab019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaaffff640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000b4a0 tx=0x7fbaa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0014070 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa0010af0 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab019d470 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0017d10 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.480+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01a4d50 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.481+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab0101d10 con 0x7fbab01018b0 2024-08-16T19:08:41.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.483+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbaa0017070 con 0x7fbab01018b0 2024-08-16T19:08:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.483+0000 7fbaad7fa640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 0x7fba980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.483+0000 7fbaaf7fe640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 0x7fba980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.483+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fbaa00cea40 con 0x7fbab01018b0 2024-08-16T19:08:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.484+0000 7fbaaf7fe640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 0x7fba980a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa4032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.486+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbaa009b760 con 0x7fbab01018b0 2024-08-16T19:08:41.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:41.700+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbab002d5e0 con 0x7fbab01018b0 2024-08-16T19:08:42.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.701+0000 7fbaad7fa640 1 -- 172.21.15.62:0/1539431464 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fbaa009b060 con 0x7fbab01018b0 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 msgr2=0x7fba980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 0x7fba980a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000c00 tx=0x7fbaa4032d50 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 msgr2=0x7fbab01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000b4a0 tx=0x7fbaa000b960 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 shutdown_connections 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fba980a2c00 0x7fba980a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbab01078b0 0x7fbab019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbab0102250 0x7fbab0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 --2- 172.21.15.62:0/1539431464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbab01018b0 0x7fbab01981d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 >> 172.21.15.62:0/1539431464 conn(0x7fbab00fd5f0 msgr2=0x7fbab00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:42.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 shutdown_connections 2024-08-16T19:08:42.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:42.704+0000 7fbab68f8640 1 -- 172.21.15.62:0/1539431464 wait complete. 2024-08-16T19:08:42.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:42 smithi062 ceph-mon[32993]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:42.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:42 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-16T19:08:43.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:42 smithi155 ceph-mon[37413]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:43.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:42 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-16T19:08:43.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:42 smithi161 ceph-mon[37576]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:43.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:42 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-16T19:08:43.376 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-08-16T19:08:43.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-08-16T19:08:43.646 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:43 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-16T19:08:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:43 smithi062 ceph-mon[32993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:44.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:43 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-16T19:08:44.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:43 smithi155 ceph-mon[37413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 -- 172.21.15.62:0/261304676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481018b0 msgr2=0x7fbf48101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/261304676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481018b0 0x7fbf48101d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009930 tx=0x7fbf4402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 -- 172.21.15.62:0/261304676 shutdown_connections 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/261304676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 0x7fbf4810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/261304676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481018b0 0x7fbf48101d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/261304676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481078b0 0x7fbf48107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 -- 172.21.15.62:0/261304676 >> 172.21.15.62:0/261304676 conn(0x7fbf480fd5f0 msgr2=0x7fbf480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:44.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 -- 172.21.15.62:0/261304676 shutdown_connections 2024-08-16T19:08:44.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.131+0000 7fbf4fd55640 1 -- 172.21.15.62:0/261304676 wait complete. 2024-08-16T19:08:44.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 Processor -- start 2024-08-16T19:08:44.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 -- start start 2024-08-16T19:08:44.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481018b0 0x7fbf48074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:44.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 0x7fbf48074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48111cb0 con 0x7fbf481078b0 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48111b30 con 0x7fbf48102250 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4fd55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbf48111e30 con 0x7fbf481018b0 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4e2f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4e2f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47736/0 (socket says 172.21.15.62:47736) 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4e2f2640 1 -- 172.21.15.62:0/2934357819 learned_addr learned my addr 172.21.15.62:0/2934357819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4daf1640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481018b0 0x7fbf48074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.132+0000 7fbf4d2f0640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 0x7fbf48074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 -- 172.21.15.62:0/2934357819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481018b0 msgr2=0x7fbf48074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481018b0 0x7fbf48074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 -- 172.21.15.62:0/2934357819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 msgr2=0x7fbf48074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:44.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 0x7fbf48074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 -- 172.21.15.62:0/2934357819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481a6630 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4e2f2640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000cce0 tx=0x7fbf4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000f450 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf481a6920 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4000fad0 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40018c10 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.133+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481a6e30 con 0x7fbf481078b0 2024-08-16T19:08:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.134+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf48101d10 con 0x7fbf481078b0 2024-08-16T19:08:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:43 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1539431464' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-16T19:08:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:43 smithi161 ceph-mon[37576]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:44.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.135+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf4000f5f0 con 0x7fbf481078b0 2024-08-16T19:08:44.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.136+0000 7fbf3effd640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 0x7fbf340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:44.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.136+0000 7fbf4daf1640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 0x7fbf340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.136+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fbf400cf820 con 0x7fbf481078b0 2024-08-16T19:08:44.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.137+0000 7fbf4daf1640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 0x7fbf340a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005fd0 tx=0x7fbf3800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:44.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.141+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbf40014030 con 0x7fbf481078b0 2024-08-16T19:08:44.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.352+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fbf4802d5e0 con 0x7fbf481078b0 2024-08-16T19:08:44.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.717+0000 7fbf3effd640 1 -- 172.21.15.62:0/2934357819 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fbf4009c540 con 0x7fbf481078b0 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 msgr2=0x7fbf340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 0x7fbf340a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005fd0 tx=0x7fbf3800b390 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 msgr2=0x7fbf48074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000cce0 tx=0x7fbf4000a620 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 shutdown_connections 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf481078b0 0x7fbf48074ca0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fbf340a2c00 0x7fbf340a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf48102250 0x7fbf48074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 --2- 172.21.15.62:0/2934357819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbf481018b0 0x7fbf48074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:44.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 >> 172.21.15.62:0/2934357819 conn(0x7fbf480fd5f0 msgr2=0x7fbf480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:44.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 shutdown_connections 2024-08-16T19:08:44.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:44.720+0000 7fbf4fd55640 1 -- 172.21.15.62:0/2934357819 wait complete. 2024-08-16T19:08:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:44 smithi062 ceph-mon[32993]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:44 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-16T19:08:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:44 smithi155 ceph-mon[37413]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:44 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-16T19:08:45.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:44 smithi161 ceph-mon[37576]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-16T19:08:45.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:44 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-08-16T19:08:45.373 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-16T19:08:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-16T19:08:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-16T19:08:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-16T19:08:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-16T19:08:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:45 smithi062 ceph-mon[32993]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-16T19:08:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:45 smithi155 ceph-mon[37413]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-16T19:08:46.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-16T19:08:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2934357819' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-16T19:08:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-16T19:08:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-16T19:08:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-16T19:08:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:45 smithi161 ceph-mon[37576]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 -- 172.21.15.62:0/2592417739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0106c80 msgr2=0x7f2fb010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/2592417739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0106c80 0x7f2fb010e820 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009cd0 tx=0x7f2fa802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 -- 172.21.15.62:0/2592417739 shutdown_connections 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/2592417739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb0106c80 0x7f2fb010e820 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/2592417739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb01009d0 0x7f2fb0106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/2592417739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0100020 0x7f2fb0100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 -- 172.21.15.62:0/2592417739 >> 172.21.15.62:0/2592417739 conn(0x7f2fb0076f40 msgr2=0x7f2fb00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 -- 172.21.15.62:0/2592417739 shutdown_connections 2024-08-16T19:08:46.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.149+0000 7f2fb76b9640 1 -- 172.21.15.62:0/2592417739 wait complete. 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 Processor -- start 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 -- start start 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb0100020 0x7f2fb0100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0106c80 0x7f2fb0101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0111cd0 con 0x7f2fb01009d0 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0111b50 con 0x7f2fb0100020 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb76b9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0111e50 con 0x7f2fb0106c80 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47760/0 (socket says 172.21.15.62:47760) 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 -- 172.21.15.62:0/1924944175 learned_addr learned my addr 172.21.15.62:0/1924944175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb5455640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb0100020 0x7f2fb0100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:46.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb5c56640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0106c80 0x7f2fb0101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 -- 172.21.15.62:0/1924944175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0106c80 msgr2=0x7f2fb0101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0106c80 0x7f2fb0101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 -- 172.21.15.62:0/1924944175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb0100020 msgr2=0x7f2fb0100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb0100020 0x7f2fb0100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.150+0000 7f2fb4c54640 1 -- 172.21.15.62:0/1924944175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01a6650 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fb4c54640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9c0 tx=0x7f2fa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000cd80 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01a6940 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0005500 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0010690 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.151+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01a6e80 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.152+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb0100400 con 0x7f2fb01009d0 2024-08-16T19:08:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.153+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2fa00046c0 con 0x7f2fb01009d0 2024-08-16T19:08:46.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.154+0000 7f2fa67fc640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:46.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.154+0000 7f2fb5455640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:46.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.154+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f2fa00cf660 con 0x7f2fb01009d0 2024-08-16T19:08:46.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.154+0000 7f2fb5455640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0058c0 tx=0x7f2fac00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:46.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.158+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2fa009c380 con 0x7f2fb01009d0 2024-08-16T19:08:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.373+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2fb01098c0 con 0x7f2fb01009d0 2024-08-16T19:08:46.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.398+0000 7f2fa67fc640 1 -- 172.21.15.62:0/1924944175 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f2fa009bc80 con 0x7f2fb01009d0 2024-08-16T19:08:46.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 msgr2=0x7f2f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:46.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0058c0 tx=0x7f2fac00a250 comp rx=0 tx=0).stop 2024-08-16T19:08:46.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 msgr2=0x7f2fb01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9c0 tx=0x7f2fa000ee80 comp rx=0 tx=0).stop 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 shutdown_connections 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fb0106c80 0x7f2fb0101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fb01009d0 0x7f2fb01014e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 --2- 172.21.15.62:0/1924944175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fb0100020 0x7f2fb0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 >> 172.21.15.62:0/1924944175 conn(0x7f2fb0076f40 msgr2=0x7f2fb0105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 shutdown_connections 2024-08-16T19:08:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:46.401+0000 7f2fb76b9640 1 -- 172.21.15.62:0/1924944175 wait complete. 2024-08-16T19:08:46.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-08-16T19:08:46.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:active 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:active 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:46.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:46 smithi062 ceph-mon[32993]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-08-16T19:08:47.059 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:active 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:active 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-08-16T19:08:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:47.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:46 smithi155 ceph-mon[37413]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] up:active 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1924944175' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-16T19:08:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] up:active 2024-08-16T19:08:47.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-16T19:08:47.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-08-16T19:08:47.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:47.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:46 smithi161 ceph-mon[37576]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-08-16T19:08:47.320 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 -- 172.21.15.62:0/4065597522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 msgr2=0x7f42f0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 --2- 172.21.15.62:0/4065597522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f0100f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e002a480 comp rx=0 tx=0).stop 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 -- 172.21.15.62:0/4065597522 shutdown_connections 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 --2- 172.21.15.62:0/4065597522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0102c20 0x7f42f010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 --2- 172.21.15.62:0/4065597522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 0x7f42f01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 --2- 172.21.15.62:0/4065597522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f0100f30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 -- 172.21.15.62:0/4065597522 >> 172.21.15.62:0/4065597522 conn(0x7f42f00fc500 msgr2=0x7f42f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 -- 172.21.15.62:0/4065597522 shutdown_connections 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.760+0000 7f42f6815640 1 -- 172.21.15.62:0/4065597522 wait complete. 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 Processor -- start 2024-08-16T19:08:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 -- start start 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 0x7f42f0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0102c20 0x7f42f019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0110cb0 con 0x7f42f0100b50 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0110b30 con 0x7f42f0102c20 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42f6815640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f42f0110e30 con 0x7f42f01022a0 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47784/0 (socket says 172.21.15.62:47784) 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 -- 172.21.15.62:0/784392593 learned_addr learned my addr 172.21.15.62:0/784392593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42e7fff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 0x7f42f0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 -- 172.21.15.62:0/784392593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 msgr2=0x7f42f0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 0x7f42f0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 -- 172.21.15.62:0/784392593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0102c20 msgr2=0x7f42f019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:47.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0102c20 0x7f42f019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.761+0000 7f42effff640 1 -- 172.21.15.62:0/784392593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f019d180 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42effff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006010 tx=0x7f42e0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e003f070 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0031e30 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f019d410 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0035030 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.762+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f01a4d50 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.763+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f0100f30 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.764+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f42e0042070 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.765+0000 7f42edffb640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 0x7f42dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.765+0000 7f42e7fff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 0x7f42dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.765+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f42e00efc00 con 0x7f42f0100b50 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.765+0000 7f42e7fff640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 0x7f42dc0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005fd0 tx=0x7f42d8005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.769+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f42e00f6080 con 0x7f42f0100b50 2024-08-16T19:08:47.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:47.983+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f42f01a4f20 con 0x7f42f0100b50 2024-08-16T19:08:48.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.417+0000 7f42edffb640 1 -- 172.21.15.62:0/784392593 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f42e00bc920 con 0x7f42f0100b50 2024-08-16T19:08:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 msgr2=0x7f42dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 0x7f42dc0a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42d8005fd0 tx=0x7f42d8005e20 comp rx=0 tx=0).stop 2024-08-16T19:08:48.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 msgr2=0x7f42f01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006010 tx=0x7f42e0003c80 comp rx=0 tx=0).stop 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 shutdown_connections 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f42dc0a2c00 0x7f42dc0a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42f0102c20 0x7f42f019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f01022a0 0x7f42f0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 --2- 172.21.15.62:0/784392593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42f0100b50 0x7f42f01981d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 >> 172.21.15.62:0/784392593 conn(0x7f42f00fc500 msgr2=0x7f42f0105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 shutdown_connections 2024-08-16T19:08:48.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:48.420+0000 7f42f6815640 1 -- 172.21.15.62:0/784392593 wait complete. 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:active 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: Cluster is now healthy 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-08-16T19:08:48.720 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:48 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:active 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: Cluster is now healthy 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-08-16T19:08:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:48 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] up:active 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: Cluster is now healthy 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-08-16T19:08:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:48 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-16T19:08:49.069 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-08-16T19:08:49.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-16T19:08:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:49 smithi062 ceph-mon[32993]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-08-16T19:08:49.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:49 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-16T19:08:49.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:49 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:active 2024-08-16T19:08:49.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:49 smithi062 ceph-mon[32993]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-08-16T19:08:49.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:49 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:08:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:49 smithi155 ceph-mon[37413]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-08-16T19:08:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:49 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-16T19:08:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:49 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:active 2024-08-16T19:08:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:49 smithi155 ceph-mon[37413]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-08-16T19:08:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:49 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:08:49.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:49 smithi161 ceph-mon[37576]: pgmap v283: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-08-16T19:08:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:49 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/784392593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-16T19:08:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:49 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] up:active 2024-08-16T19:08:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:49 smithi161 ceph-mon[37576]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-08-16T19:08:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:49 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 -- 172.21.15.62:0/4285567770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 msgr2=0x7f452810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 --2- 172.21.15.62:0/4285567770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f452810b880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4518009900 tx=0x7f451802a480 comp rx=0 tx=0).stop 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 -- 172.21.15.62:0/4285567770 shutdown_connections 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 --2- 172.21.15.62:0/4285567770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f452810bea0 0x7f452810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 --2- 172.21.15.62:0/4285567770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f452810b880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 --2- 172.21.15.62:0/4285567770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4528102a70 0x7f4528102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:49.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 -- 172.21.15.62:0/4285567770 >> 172.21.15.62:0/4285567770 conn(0x7f45280fc4d0 msgr2=0x7f45280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 -- 172.21.15.62:0/4285567770 shutdown_connections 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 -- 172.21.15.62:0/4285567770 wait complete. 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.946+0000 7f453071b640 1 Processor -- start 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 -- start start 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 0x7f4528198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 0x7f452819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4528111ca0 con 0x7f4528103390 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4528111b20 con 0x7f452810bea0 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f453071b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4528111e20 con 0x7f4528102a70 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47800/0 (socket says 172.21.15.62:47800) 2024-08-16T19:08:49.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 -- 172.21.15.62:0/2803368585 learned_addr learned my addr 172.21.15.62:0/2803368585 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452ecb8640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 0x7f452819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452e4b7640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 0x7f4528198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 -- 172.21.15.62:0/2803368585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 msgr2=0x7f4528198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 0x7f4528198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 -- 172.21.15.62:0/2803368585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 msgr2=0x7f452819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 0x7f452819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 -- 172.21.15.62:0/2803368585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452819d0c0 con 0x7f4528103390 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452ecb8640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 0x7f452819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452e4b7640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 0x7f4528198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.947+0000 7f452dcb6640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4518005ab0 tx=0x7f45180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.948+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451803f030 con 0x7f4528103390 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.948+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452819d350 con 0x7f4528103390 2024-08-16T19:08:49.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.948+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4518003ce0 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.948+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518035070 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.948+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45281a4c90 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.949+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452806ada0 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.950+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4518042030 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.951+0000 7f451f7fe640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 0x7f45140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.951+0000 7f452e4b7640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 0x7f45140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.951+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f45180efe00 con 0x7f4528103390 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.951+0000 7f452e4b7640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 0x7f45140a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45240058c0 tx=0x7f4524005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:49.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:49.954+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f45180bcb20 con 0x7f4528103390 2024-08-16T19:08:50.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.155+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4528062360 con 0x7f45140a2b80 2024-08-16T19:08:50.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.184+0000 7f451f7fe640 1 -- 172.21.15.62:0/2803368585 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4528062360 con 0x7f45140a2b80 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 msgr2=0x7f45140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 0x7f45140a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45240058c0 tx=0x7f4524005ec0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 msgr2=0x7f4528198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4518005ab0 tx=0x7f45180042f0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 shutdown_connections 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f45140a2b80 0x7f45140a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f452810bea0 0x7f452819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4528103390 0x7f4528198650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 --2- 172.21.15.62:0/2803368585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4528102a70 0x7f4528198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 >> 172.21.15.62:0/2803368585 conn(0x7f45280fc4d0 msgr2=0x7f45280fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 shutdown_connections 2024-08-16T19:08:50.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:50.186+0000 7f453071b640 1 -- 172.21.15.62:0/2803368585 wait complete. 2024-08-16T19:08:50.329 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:08:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:08:50.046+0000 7f48faf90640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-16T19:08:50.329 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:08:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:08:50.046+0000 7f48faf90640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-08-16T19:08:50.329 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:08:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:08:50.046+0000 7f48faf90640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-08-16T19:08:50.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:08:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:08:50.160+0000 7fadfe036640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:08:50.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:08:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:08:50.160+0000 7fadfe036640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:08:50.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:08:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:08:50.160+0000 7fadfe036640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:08:50.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:08:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:08:50.160+0000 7fadfe036640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:08:50.496 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:08:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:08:50.160+0000 7fadfe036640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:08:50.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-16T19:08:51.098 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:51.478 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:51 smithi062 ceph-mon[32993]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-16T19:08:51.479 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:51 smithi062 ceph-mon[32993]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1702677319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941007b0 msgr2=0x7f5b94100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1702677319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941007b0 0x7f5b94100c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009930 tx=0x7f5b7c02a420 comp rx=0 tx=0).stop 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1702677319 shutdown_connections 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1702677319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 0x7f5b9410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1702677319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941007b0 0x7f5b94100c10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1702677319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941067b0 0x7f5b94106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1702677319 >> 172.21.15.62:0/1702677319 conn(0x7f5b940fc4d0 msgr2=0x7f5b940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1702677319 shutdown_connections 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.566+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1702677319 wait complete. 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 Processor -- start 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 -- start start 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007b0 0x7f5b94197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 0x7f5b94197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94110bd0 con 0x7f5b941067b0 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94110a50 con 0x7f5b941007b0 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.567+0000 7f5b9a4ea640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94110d50 con 0x7f5b94101150 2024-08-16T19:08:51.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38990/0 (socket says 172.21.15.62:38990) 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 -- 172.21.15.62:0/1824689107 learned_addr learned my addr 172.21.15.62:0/1824689107 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b8bfff640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 0x7f5b94197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b93fff640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007b0 0x7f5b94197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 -- 172.21.15.62:0/1824689107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 msgr2=0x7f5b94197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 0x7f5b94197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 -- 172.21.15.62:0/1824689107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007b0 msgr2=0x7f5b94197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007b0 0x7f5b94197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 -- 172.21.15.62:0/1824689107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9419c0c0 con 0x7f5b941067b0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b98a87640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ebb0 tx=0x7f5b8c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00e2e0 con 0x7f5b941067b0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9419c350 con 0x7f5b941067b0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c007950 con 0x7f5b941067b0 2024-08-16T19:08:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c005790 con 0x7f5b941067b0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.568+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9406a4c0 con 0x7f5b941067b0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.569+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b94101ef0 con 0x7f5b941067b0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.571+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b8c007b00 con 0x7f5b941067b0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.572+0000 7f5b91ffb640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 0x7f5b800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.572+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f5b8c0d0c40 con 0x7f5b941067b0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.572+0000 7f5b93fff640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 0x7f5b800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.573+0000 7f5b93fff640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 0x7f5b800a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009990 tx=0x7f5b84009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:51.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.575+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5b8c09d960 con 0x7f5b941067b0 2024-08-16T19:08:51.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.780+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5b941984d0 con 0x7f5b800a2b80 2024-08-16T19:08:51.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.806+0000 7f5b91ffb640 1 -- 172.21.15.62:0/1824689107 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b941984d0 con 0x7f5b800a2b80 2024-08-16T19:08:51.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.807+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 msgr2=0x7f5b800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:51.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.807+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 0x7f5b800a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009990 tx=0x7f5b84009110 comp rx=0 tx=0).stop 2024-08-16T19:08:51.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.807+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 msgr2=0x7f5b9419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:51.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.807+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00ebb0 tx=0x7f5b8c009f60 comp rx=0 tx=0).stop 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 shutdown_connections 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b941067b0 0x7f5b9419b9e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b94101150 0x7f5b94197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f5b800a2b80 0x7f5b800a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 --2- 172.21.15.62:0/1824689107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b941007b0 0x7f5b94197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 >> 172.21.15.62:0/1824689107 conn(0x7f5b940fc4d0 msgr2=0x7f5b940fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 shutdown_connections 2024-08-16T19:08:51.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:51.808+0000 7f5b9a4ea640 1 -- 172.21.15.62:0/1824689107 wait complete. 2024-08-16T19:08:51.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:51 smithi155 ceph-mon[37413]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-16T19:08:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:51 smithi155 ceph-mon[37413]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:51 smithi161 ceph-mon[37576]: pgmap v284: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-08-16T19:08:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:51 smithi161 ceph-mon[37576]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:52.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-16T19:08:52.707 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:53.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 -- 172.21.15.62:0/3458931361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1018b0 msgr2=0x7f799c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:53.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 --2- 172.21.15.62:0/3458931361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1018b0 0x7f799c101d10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7984009930 tx=0x7f798402a420 comp rx=0 tx=0).stop 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 -- 172.21.15.62:0/3458931361 shutdown_connections 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 --2- 172.21.15.62:0/3458931361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c102250 0x7f799c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 --2- 172.21.15.62:0/3458931361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1018b0 0x7f799c101d10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 --2- 172.21.15.62:0/3458931361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c1078b0 0x7f799c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 -- 172.21.15.62:0/3458931361 >> 172.21.15.62:0/3458931361 conn(0x7f799c0fd5f0 msgr2=0x7f799c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 -- 172.21.15.62:0/3458931361 shutdown_connections 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.173+0000 7f79a1b73640 1 -- 172.21.15.62:0/3458931361 wait complete. 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 Processor -- start 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 -- start start 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 0x7f799c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c102250 0x7f799c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:53.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f799c111cb0 con 0x7f799c1078b0 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f799c111b30 con 0x7f799c1018b0 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f79a1b73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f799c111e30 con 0x7f799c102250 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f799bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f799bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38998/0 (socket says 172.21.15.62:38998) 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f799bfff640 1 -- 172.21.15.62:0/2983337197 learned_addr learned my addr 172.21.15.62:0/2983337197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.174+0000 7f7993fff640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c102250 0x7f799c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799b7fe640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 0x7f799c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 -- 172.21.15.62:0/2983337197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c102250 msgr2=0x7f799c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c102250 0x7f799c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 -- 172.21.15.62:0/2983337197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 msgr2=0x7f799c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:53.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 0x7f799c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 -- 172.21.15.62:0/2983337197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c19d120 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799b7fe640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 0x7f799c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f799bfff640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f799400cce0 tx=0x7f799400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799400f450 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799400fad0 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c19d410 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994018c10 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.175+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a4cf0 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.176+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c101d10 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.177+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f799400f5f0 con 0x7f799c1078b0 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.178+0000 7f79997fa640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 0x7f79880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:53.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.178+0000 7f799b7fe640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 0x7f79880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:53.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.179+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f79940cfc70 con 0x7f799c1078b0 2024-08-16T19:08:53.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.179+0000 7f799b7fe640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 0x7f79880a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f798c005fd0 tx=0x7f798c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:53.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.182+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7994014030 con 0x7f799c1078b0 2024-08-16T19:08:53.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.385+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f799c062360 con 0x7f79880a2c10 2024-08-16T19:08:53.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.390+0000 7f79997fa640 1 -- 172.21.15.62:0/2983337197 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f799c062360 con 0x7f79880a2c10 2024-08-16T19:08:53.391 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_0/8bf3ca03-9e94-4850-b6bd-e3449de64c1c 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 msgr2=0x7f79880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 0x7f79880a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f798c005fd0 tx=0x7f798c0094d0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 msgr2=0x7f799c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f799400cce0 tx=0x7f799400a620 comp rx=0 tx=0).stop 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 shutdown_connections 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f799c1078b0 0x7f799c19ca40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f799c102250 0x7f799c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f79880a2c10 0x7f79880a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 --2- 172.21.15.62:0/2983337197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f799c1018b0 0x7f799c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:53.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 >> 172.21.15.62:0/2983337197 conn(0x7f799c0fd5f0 msgr2=0x7f799c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:53.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 shutdown_connections 2024-08-16T19:08:53.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:53.392+0000 7f79a1b73640 1 -- 172.21.15.62:0/2983337197 wait complete. 2024-08-16T19:08:53.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:53 smithi161 ceph-mon[37576]: pgmap v285: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-08-16T19:08:53.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:53 smithi161 ceph-mon[37576]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:53.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:53 smithi161 ceph-mon[37576]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-16T19:08:53.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:53 smithi062 ceph-mon[32993]: pgmap v285: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-08-16T19:08:53.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:53 smithi062 ceph-mon[32993]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:53.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:53 smithi062 ceph-mon[32993]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-16T19:08:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:53 smithi155 ceph-mon[37413]: pgmap v285: 129 pgs: 129 active+clean; 587 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-08-16T19:08:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:53 smithi155 ceph-mon[37413]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:53 smithi155 ceph-mon[37413]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-16T19:08:54.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-16T19:08:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:54.702 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:54 smithi062 ceph-mon[32993]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:54.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:54 smithi155 ceph-mon[37413]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:54.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:54 smithi161 ceph-mon[37576]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- 172.21.15.62:0/3886278755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 msgr2=0x7f62481026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 --2- 172.21.15.62:0/3886278755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481026e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6238009900 tx=0x7f623802a480 comp rx=0 tx=0).stop 2024-08-16T19:08:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- 172.21.15.62:0/3886278755 shutdown_connections 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 --2- 172.21.15.62:0/3886278755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 --2- 172.21.15.62:0/3886278755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481026e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 --2- 172.21.15.62:0/3886278755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 0x7f6248100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- 172.21.15.62:0/3886278755 >> 172.21.15.62:0/3886278755 conn(0x7f62480fc500 msgr2=0x7f62480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- 172.21.15.62:0/3886278755 shutdown_connections 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- 172.21.15.62:0/3886278755 wait complete. 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 Processor -- start 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.885+0000 7f6250694640 1 -- start start 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 0x7f62481982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6248110cb0 con 0x7f62481022a0 2024-08-16T19:08:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6248110b30 con 0x7f6248102c20 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6248110e30 con 0x7f6248100b50 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39010/0 (socket says 172.21.15.62:39010) 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 -- 172.21.15.62:0/3841990656 learned_addr learned my addr 172.21.15.62:0/3841990656 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624e430640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 0x7f62481982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624ec31640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 -- 172.21.15.62:0/3841990656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 msgr2=0x7f62481982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 0x7f62481982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 -- 172.21.15.62:0/3841990656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 msgr2=0x7f624819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 -- 172.21.15.62:0/3841990656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624819d250 con 0x7f62481022a0 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624ec31640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f624dc2f640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6238006010 tx=0x7f6238003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:54.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623803f070 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6238031c30 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624819d4e0 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.886+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238035030 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.887+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481a4e20 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.888+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6248100f30 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.889+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6238042070 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.889+0000 7f623f7fe640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 0x7f62340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.890+0000 7f624e430640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 0x7f62340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.890+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f62380f0020 con 0x7f62481022a0 2024-08-16T19:08:54.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.890+0000 7f624e430640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 0x7f62340a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f624400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:54.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:54.893+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f62380bcbe0 con 0x7f62481022a0 2024-08-16T19:08:55.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.095+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6248062360 con 0x7f62340a2c90 2024-08-16T19:08:55.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.120+0000 7f623f7fe640 1 -- 172.21.15.62:0/3841990656 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6248062360 con 0x7f62340a2c90 2024-08-16T19:08:55.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 msgr2=0x7f62340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:55.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 0x7f62340a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f624400b390 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 msgr2=0x7f62481987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6238006010 tx=0x7f6238003c10 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 shutdown_connections 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f62340a2c90 0x7f62340a5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6248102c20 0x7f624819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62481022a0 0x7f62481987e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 --2- 172.21.15.62:0/3841990656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6248100b50 0x7f62481982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 >> 172.21.15.62:0/3841990656 conn(0x7f62480fc500 msgr2=0x7f624806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 shutdown_connections 2024-08-16T19:08:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:55.122+0000 7f6250694640 1 -- 172.21.15.62:0/3841990656 wait complete. 2024-08-16T19:08:55.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:55 smithi062 ceph-mon[32993]: pgmap v286: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-08-16T19:08:55.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-16T19:08:55.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:55 smithi155 ceph-mon[37413]: pgmap v286: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-08-16T19:08:55.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:55 smithi161 ceph-mon[37576]: pgmap v286: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-08-16T19:08:56.067 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:56.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 -- 172.21.15.62:0/4255859303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 msgr2=0x7f4a3c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 --2- 172.21.15.62:0/4255859303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c10e340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009ca0 tx=0x7f4a3402a950 comp rx=0 tx=0).stop 2024-08-16T19:08:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 -- 172.21.15.62:0/4255859303 shutdown_connections 2024-08-16T19:08:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 --2- 172.21.15.62:0/4255859303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c10e340 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 --2- 172.21.15.62:0/4255859303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c103390 0x7f4a3c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.529+0000 7f4a4190a640 1 --2- 172.21.15.62:0/4255859303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c102a70 0x7f4a3c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- 172.21.15.62:0/4255859303 >> 172.21.15.62:0/4255859303 conn(0x7f4a3c0fc4d0 msgr2=0x7f4a3c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- 172.21.15.62:0/4255859303 shutdown_connections 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- 172.21.15.62:0/4255859303 wait complete. 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 Processor -- start 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- start start 2024-08-16T19:08:56.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c102a70 0x7f4a3c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 0x7f4a3c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c111ca0 con 0x7f4a3c10bea0 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c111b20 con 0x7f4a3c103390 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.530+0000 7f4a4190a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c111e20 con 0x7f4a3c102a70 2024-08-16T19:08:56.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39026/0 (socket says 172.21.15.62:39026) 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 -- 172.21.15.62:0/1273948662 learned_addr learned my addr 172.21.15.62:0/1273948662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a33fff640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 0x7f4a3c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3affd640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c102a70 0x7f4a3c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 -- 172.21.15.62:0/1273948662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c102a70 msgr2=0x7f4a3c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c102a70 0x7f4a3c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 -- 172.21.15.62:0/1273948662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 msgr2=0x7f4a3c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 0x7f4a3c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 -- 172.21.15.62:0/1273948662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c19d0b0 con 0x7f4a3c10bea0 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a33fff640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 0x7f4a3c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a3b7fe640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a34004af0 tx=0x7f4a3402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34034040 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a34030920 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c19d340 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340047b0 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.531+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a4c80 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.532+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c102e50 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.534+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a34037020 con 0x7f4a3c10bea0 2024-08-16T19:08:56.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.534+0000 7f4a40908640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.535+0000 7f4a3affd640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.535+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f4a340effe0 con 0x7f4a3c10bea0 2024-08-16T19:08:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.535+0000 7f4a3affd640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 0x7f4a280a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009bb0 tx=0x7f4a2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:56.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.538+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4a340bcc90 con 0x7f4a3c10bea0 2024-08-16T19:08:56.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.740+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4a3c106f80 con 0x7f4a280a2c90 2024-08-16T19:08:56.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:56 smithi062 ceph-mon[32993]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:56.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.745+0000 7f4a40908640 1 -- 172.21.15.62:0/1273948662 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4a2c004030 con 0x7f4a280a2c90 2024-08-16T19:08:56.746 INFO:teuthology.orchestra.run.smithi062.stdout:/volumes/qa/sv_1/5dce479e-2db7-4669-b90f-338aacf1c0d8 2024-08-16T19:08:56.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 msgr2=0x7f4a280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:56.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 0x7f4a280a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009bb0 tx=0x7f4a2c005f90 comp rx=0 tx=0).stop 2024-08-16T19:08:56.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 msgr2=0x7f4a3c19c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:56.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a34004af0 tx=0x7f4a3402f5b0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 shutdown_connections 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a3c10bea0 0x7f4a3c19c9d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c103390 0x7f4a3c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 --2- 172.21.15.62:0/1273948662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a3c102a70 0x7f4a3c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 >> 172.21.15.62:0/1273948662 conn(0x7f4a3c0fc4d0 msgr2=0x7f4a3c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:56.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 shutdown_connections 2024-08-16T19:08:56.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:56.747+0000 7f4a4190a640 1 -- 172.21.15.62:0/1273948662 wait complete. 2024-08-16T19:08:56.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:56 smithi155 ceph-mon[37413]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:56.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:56 smithi161 ceph-mon[37576]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:57.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:08:57.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:57 smithi062 ceph-mon[32993]: pgmap v287: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-08-16T19:08:57.790 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:57 smithi155 ceph-mon[37413]: pgmap v287: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-08-16T19:08:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:57 smithi161 ceph-mon[37576]: pgmap v287: 129 pgs: 129 active+clean; 608 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.7 KiB/s wr, 22 op/s 2024-08-16T19:08:58.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 -- 172.21.15.62:0/2441714292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41018b0 msgr2=0x7f06e4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 --2- 172.21.15.62:0/2441714292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41018b0 0x7f06e4101d10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009930 tx=0x7f06d802a420 comp rx=0 tx=0).stop 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 -- 172.21.15.62:0/2441714292 shutdown_connections 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 --2- 172.21.15.62:0/2441714292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 0x7f06e410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 --2- 172.21.15.62:0/2441714292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41018b0 0x7f06e4101d10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 --2- 172.21.15.62:0/2441714292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41078b0 0x7f06e4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 -- 172.21.15.62:0/2441714292 >> 172.21.15.62:0/2441714292 conn(0x7f06e40fd5f0 msgr2=0x7f06e40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 -- 172.21.15.62:0/2441714292 shutdown_connections 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.192+0000 7f06e9c20640 1 -- 172.21.15.62:0/2441714292 wait complete. 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.193+0000 7f06e9c20640 1 Processor -- start 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.193+0000 7f06e9c20640 1 -- start start 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41018b0 0x7f06e4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 0x7f06e4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4111c70 con 0x7f06e41078b0 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4111af0 con 0x7f06e41018b0 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06e4111df0 con 0x7f06e4102250 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39042/0 (socket says 172.21.15.62:39042) 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 -- 172.21.15.62:0/1725861313 learned_addr learned my addr 172.21.15.62:0/1725861313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e2ffd640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 0x7f06e4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e37fe640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41018b0 0x7f06e4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 -- 172.21.15.62:0/1725861313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 msgr2=0x7f06e4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 0x7f06e4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 -- 172.21.15.62:0/1725861313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41018b0 msgr2=0x7f06e4074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41018b0 0x7f06e4074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 -- 172.21.15.62:0/1725861313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e41a6610 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e3fff640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00cad0 tx=0x7f06dc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00ad60 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.194+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e41a6900 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.195+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06dc005020 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.195+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc00f6b0 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.195+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e41a6dc0 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.196+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e4101d10 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.197+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f06dc005560 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.197+0000 7f06e0ff9640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.198+0000 7f06e37fe640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.198+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f06dc0cf400 con 0x7f06e41078b0 2024-08-16T19:08:58.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.198+0000 7f06e37fe640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06d400ab60 tx=0x7f06d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:58.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.202+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f06dc014030 con 0x7f06e41078b0 2024-08-16T19:08:58.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.415+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06e41a7080 con 0x7f06e41078b0 2024-08-16T19:08:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.415+0000 7f06e0ff9640 1 -- 172.21.15.62:0/1725861313 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f06dc09bfc0 con 0x7f06e41078b0 2024-08-16T19:08:58.416 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:08:58.417 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:08:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.417+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 msgr2=0x7f06cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.417+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06d400ab60 tx=0x7f06d400a2d0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.417+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 msgr2=0x7f06e4074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.417+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00cad0 tx=0x7f06dc00ced0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 shutdown_connections 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e41078b0 0x7f06e4074ca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06e4102250 0x7f06e4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 --2- 172.21.15.62:0/1725861313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06e41018b0 0x7f06e4074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 >> 172.21.15.62:0/1725861313 conn(0x7f06e40fd5f0 msgr2=0x7f06e40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 shutdown_connections 2024-08-16T19:08:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:58.418+0000 7f06e9c20640 1 -- 172.21.15.62:0/1725861313 wait complete. 2024-08-16T19:08:58.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:58 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1725861313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-08-16T19:08:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-16T19:08:58.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-08-16T19:08:58.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:58 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1725861313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-16T19:08:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-08-16T19:08:58.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:58 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1725861313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:08:59.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:08:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:08:59.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: pgmap v288: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 25 op/s 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-08-16T19:08:59.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:08:59 smithi062 ceph-mon[32993]: osdmap e89: 12 total, 12 up, 12 in 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: pgmap v288: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 25 op/s 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-08-16T19:08:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-16T19:08:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-16T19:08:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-08-16T19:08:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-08-16T19:08:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-08-16T19:08:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:08:59 smithi155 ceph-mon[37413]: osdmap e89: 12 total, 12 up, 12 in 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 -- 172.21.15.62:0/2898595955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 msgr2=0x7f3bf0107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/2898595955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf0107c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009930 tx=0x7f3bec02a420 comp rx=0 tx=0).stop 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 -- 172.21.15.62:0/2898595955 shutdown_connections 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/2898595955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf0102250 0x7f3bf010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/2898595955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf01018b0 0x7f3bf0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:59.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/2898595955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf0107c90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 -- 172.21.15.62:0/2898595955 >> 172.21.15.62:0/2898595955 conn(0x7f3bf00fd5f0 msgr2=0x7f3bf00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 -- 172.21.15.62:0/2898595955 shutdown_connections 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.834+0000 7f3bf79a4640 1 -- 172.21.15.62:0/2898595955 wait complete. 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 Processor -- start 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 -- start start 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf01018b0 0x7f3bf0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf0102250 0x7f3bf0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111d00 con 0x7f3bf01078b0 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111b80 con 0x7f3bf0102250 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.835+0000 7f3bf79a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111e80 con 0x7f3bf01018b0 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5740640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf01018b0 0x7f3bf0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39058/0 (socket says 172.21.15.62:39058) 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 -- 172.21.15.62:0/704223909 learned_addr learned my addr 172.21.15.62:0/704223909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf4f3f640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf0102250 0x7f3bf0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 -- 172.21.15.62:0/704223909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf01018b0 msgr2=0x7f3bf0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf01018b0 0x7f3bf0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 -- 172.21.15.62:0/704223909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf0102250 msgr2=0x7f3bf0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf0102250 0x7f3bf0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 -- 172.21.15.62:0/704223909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf019d0e0 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf5f41640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3be800cce0 tx=0x7f3be800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be800f450 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf019d3d0 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be800fad0 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8018c10 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.836+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01a4cb0 con 0x7f3bf01078b0 2024-08-16T19:08:59.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.837+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf0101d10 con 0x7f3bf01078b0 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.839+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3be800f5f0 con 0x7f3bf01078b0 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.840+0000 7f3be27fc640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 0x7f3bdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.840+0000 7f3bf5740640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 0x7f3bdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.840+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3be80cfb10 con 0x7f3bf01078b0 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.840+0000 7f3bf5740640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 0x7f3bdc0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3bec006010 tx=0x7f3bec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:08:59.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:08:59.844+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3be8014030 con 0x7f3bf01078b0 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: pgmap v288: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 25 op/s 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-08-16T19:08:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-08-16T19:08:59.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:08:59 smithi161 ceph-mon[37576]: osdmap e89: 12 total, 12 up, 12 in 2024-08-16T19:09:00.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.021+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bf0109890 con 0x7f3bf01078b0 2024-08-16T19:09:00.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.022+0000 7f3be27fc640 1 -- 172.21.15.62:0/704223909 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24632 (secure 0 0 0) 0x7f3be809c550 con 0x7f3bf01078b0 2024-08-16T19:09:00.023 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:00.023 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:59.449917+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 msgr2=0x7f3bdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 0x7f3bdc0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3bec006010 tx=0x7f3bec000bc0 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 msgr2=0x7f3bf019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3be800cce0 tx=0x7f3be800a620 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 shutdown_connections 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bf01078b0 0x7f3bf019ca00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bdc0a2c10 0x7f3bdc0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bf0102250 0x7f3bf0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 --2- 172.21.15.62:0/704223909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bf01018b0 0x7f3bf0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:00.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 >> 172.21.15.62:0/704223909 conn(0x7f3bf00fd5f0 msgr2=0x7f3bf00feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:00.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.024+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 shutdown_connections 2024-08-16T19:09:00.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:00.025+0000 7f3bf79a4640 1 -- 172.21.15.62:0/704223909 wait complete. 2024-08-16T19:09:00.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:00 smithi062 ceph-mon[32993]: osdmap e90: 12 total, 12 up, 12 in 2024-08-16T19:09:00.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:00 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/704223909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:00.627 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:00 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:09:00.660 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-16T19:09:00.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:09:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:00 smithi155 ceph-mon[37413]: osdmap e90: 12 total, 12 up, 12 in 2024-08-16T19:09:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:00 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/704223909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:00 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:09:00.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:00 smithi161 ceph-mon[37576]: osdmap e90: 12 total, 12 up, 12 in 2024-08-16T19:09:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:00 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/704223909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:00 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:09:01.585 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:01 smithi062 ceph-mon[32993]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-08-16T19:09:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:01 smithi155 ceph-mon[37413]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-08-16T19:09:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:01 smithi161 ceph-mon[37576]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 -- 172.21.15.62:0/799840230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc168102250 msgr2=0x7fc16810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 --2- 172.21.15.62:0/799840230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc168102250 0x7fc16810e780 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc160009cd0 tx=0x7fc16002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 -- 172.21.15.62:0/799840230 shutdown_connections 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 --2- 172.21.15.62:0/799840230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc168102250 0x7fc16810e780 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 --2- 172.21.15.62:0/799840230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 0x7fc168101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 --2- 172.21.15.62:0/799840230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1681078b0 0x7fc168107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 -- 172.21.15.62:0/799840230 >> 172.21.15.62:0/799840230 conn(0x7fc1680fd5f0 msgr2=0x7fc1680ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 -- 172.21.15.62:0/799840230 shutdown_connections 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.043+0000 7fc16fc93640 1 -- 172.21.15.62:0/799840230 wait complete. 2024-08-16T19:09:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 Processor -- start 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 -- start start 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 0x7fc168198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc168102250 0x7fc1681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111cb0 con 0x7fc1681078b0 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111b30 con 0x7fc1681018b0 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16fc93640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc168111e30 con 0x7fc168102250 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50408/0 (socket says 172.21.15.62:50408) 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16da2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 0x7fc168198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 -- 172.21.15.62:0/868576021 learned_addr learned my addr 172.21.15.62:0/868576021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16d22e640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc168102250 0x7fc1681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 -- 172.21.15.62:0/868576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc168102250 msgr2=0x7fc1681986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc168102250 0x7fc1681986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 -- 172.21.15.62:0/868576021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 msgr2=0x7fc168198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 0x7fc168198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.044+0000 7fc16e230640 1 -- 172.21.15.62:0/868576021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16819d110 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc16e230640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc160005cc0 tx=0x7fc160003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160004040 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16819d3a0 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1600046c0 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160046480 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.045+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1681a4ce0 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.046+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc168101d10 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.047+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1600305a0 con 0x7fc1681078b0 2024-08-16T19:09:02.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.048+0000 7fc15affd640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 0x7fc1540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.048+0000 7fc16da2f640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 0x7fc1540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.048+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fc16002e030 con 0x7fc1681078b0 2024-08-16T19:09:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.049+0000 7fc16da2f640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 0x7fc1540a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005fd0 tx=0x7fc15c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.052+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc16004b060 con 0x7fc1681078b0 2024-08-16T19:09:02.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.241+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc168062360 con 0x7fc1540a2c10 2024-08-16T19:09:02.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.251+0000 7fc15affd640 1 -- 172.21.15.62:0/868576021 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340417 (secure 0 0 0) 0x7fc168062360 con 0x7fc1540a2c10 2024-08-16T19:09:02.256 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:02.257 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 msgr2=0x7fc1540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 0x7fc1540a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005fd0 tx=0x7fc15c00b390 comp rx=0 tx=0).stop 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 msgr2=0x7fc16819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc160005cc0 tx=0x7fc160003970 comp rx=0 tx=0).stop 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 shutdown_connections 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1681078b0 0x7fc16819ca30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc168102250 0x7fc1681986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1540a2c10 0x7fc1540a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 --2- 172.21.15.62:0/868576021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1681018b0 0x7fc168198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:02.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 >> 172.21.15.62:0/868576021 conn(0x7fc1680fd5f0 msgr2=0x7fc1680fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:02.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 shutdown_connections 2024-08-16T19:09:02.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:02.254+0000 7fc16fc93640 1 -- 172.21.15.62:0/868576021 wait complete. 2024-08-16T19:09:03.004 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-08-16T19:09:01.659285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641533,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":290,"num_read_kb":252,"num_write":421,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375752,"kb_used_data":14064,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566072,"statfs":{"total":1151940427776,"available":1151555657728,"internally_reserved":0,"allocated":14401536,"data_stored":7934201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19389,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":65,"num_write":21,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001754"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457998+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:08:59.457998+0000","last_peered":"2024-08-16T19:08:59.457998+0000","last_clean":"2024-08-16T19:08:59.457998+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:08:59.457998+0000","last_undegraded":"2024-08-16T19:08:59.457998+0000","last_fullsized":"2024-08-16T19:08:59.457998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.461987+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:08:58.461987+0000","last_peered":"2024-08-16T19:08:58.461987+0000","last_clean":"2024-08-16T19:08:58.461987+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:08:58.461987+0000","last_undegraded":"2024-08-16T19:08:58.461987+0000","last_fullsized":"2024-08-16T19:08:58.461987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806508+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:08:14.806508+0000","last_peered":"2024-08-16T19:08:14.806508+0000","last_clean":"2024-08-16T19:08:14.806508+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:08:14.806508+0000","last_undegraded":"2024-08-16T19:08:14.806508+0000","last_fullsized":"2024-08-16T19:08:14.806508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.462263+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:08:58.462263+0000","last_peered":"2024-08-16T19:08:58.462263+0000","last_clean":"2024-08-16T19:08:58.462263+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:08:58.462263+0000","last_undegraded":"2024-08-16T19:08:58.462263+0000","last_fullsized":"2024-08-16T19:08:58.462263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458615+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:08:59.458615+0000","last_peered":"2024-08-16T19:08:59.458615+0000","last_clean":"2024-08-16T19:08:59.458615+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:08:59.458615+0000","last_undegraded":"2024-08-16T19:08:59.458615+0000","last_fullsized":"2024-08-16T19:08:59.458615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.594556+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:08:59.594556+0000","last_peered":"2024-08-16T19:08:59.594556+0000","last_clean":"2024-08-16T19:08:59.594556+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:08:59.594556+0000","last_undegraded":"2024-08-16T19:08:59.594556+0000","last_fullsized":"2024-08-16T19:08:59.594556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459473+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:08:59.459473+0000","last_peered":"2024-08-16T19:08:59.459473+0000","last_clean":"2024-08-16T19:08:59.459473+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:08:59.459473+0000","last_undegraded":"2024-08-16T19:08:59.459473+0000","last_fullsized":"2024-08-16T19:08:59.459473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545800+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:08:59.545800+0000","last_peered":"2024-08-16T19:08:59.545800+0000","last_clean":"2024-08-16T19:08:59.545800+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:08:59.545800+0000","last_undegraded":"2024-08-16T19:08:59.545800+0000","last_fullsized":"2024-08-16T19:08:59.545800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457100+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:08:59.457100+0000","last_peered":"2024-08-16T19:08:59.457100+0000","last_clean":"2024-08-16T19:08:59.457100+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:08:59.457100+0000","last_undegraded":"2024-08-16T19:08:59.457100+0000","last_fullsized":"2024-08-16T19:08:59.457100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459443+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:08:59.459443+0000","last_peered":"2024-08-16T19:08:59.459443+0000","last_clean":"2024-08-16T19:08:59.459443+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:08:59.459443+0000","last_undegraded":"2024-08-16T19:08:59.459443+0000","last_fullsized":"2024-08-16T19:08:59.459443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.547859+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:08:59.547859+0000","last_peered":"2024-08-16T19:08:59.547859+0000","last_clean":"2024-08-16T19:08:59.547859+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:08:59.547859+0000","last_undegraded":"2024-08-16T19:08:59.547859+0000","last_fullsized":"2024-08-16T19:08:59.547859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457189+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:08:59.457189+0000","last_peered":"2024-08-16T19:08:59.457189+0000","last_clean":"2024-08-16T19:08:59.457189+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:08:59.457189+0000","last_undegraded":"2024-08-16T19:08:59.457189+0000","last_fullsized":"2024-08-16T19:08:59.457189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454902+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:08:59.454902+0000","last_peered":"2024-08-16T19:08:59.454902+0000","last_clean":"2024-08-16T19:08:59.454902+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:08:59.454902+0000","last_undegraded":"2024-08-16T19:08:59.454902+0000","last_fullsized":"2024-08-16T19:08:59.454902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458666+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:08:59.458666+0000","last_peered":"2024-08-16T19:08:59.458666+0000","last_clean":"2024-08-16T19:08:59.458666+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:08:59.458666+0000","last_undegraded":"2024-08-16T19:08:59.458666+0000","last_fullsized":"2024-08-16T19:08:59.458666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457164+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:08:59.457164+0000","last_peered":"2024-08-16T19:08:59.457164+0000","last_clean":"2024-08-16T19:08:59.457164+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:08:59.457164+0000","last_undegraded":"2024-08-16T19:08:59.457164+0000","last_fullsized":"2024-08-16T19:08:59.457164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.460945+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:08:58.460945+0000","last_peered":"2024-08-16T19:08:58.460945+0000","last_clean":"2024-08-16T19:08:58.460945+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:08:58.460945+0000","last_undegraded":"2024-08-16T19:08:58.460945+0000","last_fullsized":"2024-08-16T19:08:58.460945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:46.730464+0000","last_change":"2024-08-16T19:08:02.462372+0000","last_active":"2024-08-16T19:08:46.730464+0000","last_peered":"2024-08-16T19:08:46.730464+0000","last_clean":"2024-08-16T19:08:46.730464+0000","last_became_active":"2024-08-16T19:08:02.461973+0000","last_became_peered":"2024-08-16T19:08:02.461973+0000","last_unstale":"2024-08-16T19:08:46.730464+0000","last_undegraded":"2024-08-16T19:08:46.730464+0000","last_fullsized":"2024-08-16T19:08:46.730464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:24:04.191300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457135+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:08:59.457135+0000","last_peered":"2024-08-16T19:08:59.457135+0000","last_clean":"2024-08-16T19:08:59.457135+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:08:59.457135+0000","last_undegraded":"2024-08-16T19:08:59.457135+0000","last_fullsized":"2024-08-16T19:08:59.457135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462630+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:08:59.462630+0000","last_peered":"2024-08-16T19:08:59.462630+0000","last_clean":"2024-08-16T19:08:59.462630+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:08:59.462630+0000","last_undegraded":"2024-08-16T19:08:59.462630+0000","last_fullsized":"2024-08-16T19:08:59.462630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.459835+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:08:58.459835+0000","last_peered":"2024-08-16T19:08:58.459835+0000","last_clean":"2024-08-16T19:08:58.459835+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:08:58.459835+0000","last_undegraded":"2024-08-16T19:08:58.459835+0000","last_fullsized":"2024-08-16T19:08:58.459835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457650+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:08:59.457650+0000","last_peered":"2024-08-16T19:08:59.457650+0000","last_clean":"2024-08-16T19:08:59.457650+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:08:59.457650+0000","last_undegraded":"2024-08-16T19:08:59.457650+0000","last_fullsized":"2024-08-16T19:08:59.457650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458319+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:08:59.458319+0000","last_peered":"2024-08-16T19:08:59.458319+0000","last_clean":"2024-08-16T19:08:59.458319+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:08:59.458319+0000","last_undegraded":"2024-08-16T19:08:59.458319+0000","last_fullsized":"2024-08-16T19:08:59.458319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545351+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:08:59.545351+0000","last_peered":"2024-08-16T19:08:59.545351+0000","last_clean":"2024-08-16T19:08:59.545351+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:08:59.545351+0000","last_undegraded":"2024-08-16T19:08:59.545351+0000","last_fullsized":"2024-08-16T19:08:59.545351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.464030+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:08:59.464030+0000","last_peered":"2024-08-16T19:08:59.464030+0000","last_clean":"2024-08-16T19:08:59.464030+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:08:59.464030+0000","last_undegraded":"2024-08-16T19:08:59.464030+0000","last_fullsized":"2024-08-16T19:08:59.464030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458222+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:08:59.458222+0000","last_peered":"2024-08-16T19:08:59.458222+0000","last_clean":"2024-08-16T19:08:59.458222+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:08:59.458222+0000","last_undegraded":"2024-08-16T19:08:59.458222+0000","last_fullsized":"2024-08-16T19:08:59.458222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458400+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:08:59.458400+0000","last_peered":"2024-08-16T19:08:59.458400+0000","last_clean":"2024-08-16T19:08:59.458400+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:08:59.458400+0000","last_undegraded":"2024-08-16T19:08:59.458400+0000","last_fullsized":"2024-08-16T19:08:59.458400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806682+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:08:14.806682+0000","last_peered":"2024-08-16T19:08:14.806682+0000","last_clean":"2024-08-16T19:08:14.806682+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:08:14.806682+0000","last_undegraded":"2024-08-16T19:08:14.806682+0000","last_fullsized":"2024-08-16T19:08:14.806682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466015+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:08:59.466015+0000","last_peered":"2024-08-16T19:08:59.466015+0000","last_clean":"2024-08-16T19:08:59.466015+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:08:59.466015+0000","last_undegraded":"2024-08-16T19:08:59.466015+0000","last_fullsized":"2024-08-16T19:08:59.466015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456637+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:08:59.456637+0000","last_peered":"2024-08-16T19:08:59.456637+0000","last_clean":"2024-08-16T19:08:59.456637+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:08:59.456637+0000","last_undegraded":"2024-08-16T19:08:59.456637+0000","last_fullsized":"2024-08-16T19:08:59.456637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.460491+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:08:58.460491+0000","last_peered":"2024-08-16T19:08:58.460491+0000","last_clean":"2024-08-16T19:08:58.460491+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:08:58.460491+0000","last_undegraded":"2024-08-16T19:08:58.460491+0000","last_fullsized":"2024-08-16T19:08:58.460491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806171+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:08:14.806171+0000","last_peered":"2024-08-16T19:08:14.806171+0000","last_clean":"2024-08-16T19:08:14.806171+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:08:14.806171+0000","last_undegraded":"2024-08-16T19:08:14.806171+0000","last_fullsized":"2024-08-16T19:08:14.806171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459679+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:08:59.459679+0000","last_peered":"2024-08-16T19:08:59.459679+0000","last_clean":"2024-08-16T19:08:59.459679+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:08:59.459679+0000","last_undegraded":"2024-08-16T19:08:59.459679+0000","last_fullsized":"2024-08-16T19:08:59.459679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462351+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:08:59.462351+0000","last_peered":"2024-08-16T19:08:59.462351+0000","last_clean":"2024-08-16T19:08:59.462351+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:08:59.462351+0000","last_undegraded":"2024-08-16T19:08:59.462351+0000","last_fullsized":"2024-08-16T19:08:59.462351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806874+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:08:14.806874+0000","last_peered":"2024-08-16T19:08:14.806874+0000","last_clean":"2024-08-16T19:08:14.806874+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:08:14.806874+0000","last_undegraded":"2024-08-16T19:08:14.806874+0000","last_fullsized":"2024-08-16T19:08:14.806874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456378+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:08:59.456378+0000","last_peered":"2024-08-16T19:08:59.456378+0000","last_clean":"2024-08-16T19:08:59.456378+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:08:59.456378+0000","last_undegraded":"2024-08-16T19:08:59.456378+0000","last_fullsized":"2024-08-16T19:08:59.456378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458335+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:08:59.458335+0000","last_peered":"2024-08-16T19:08:59.458335+0000","last_clean":"2024-08-16T19:08:59.458335+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:08:59.458335+0000","last_undegraded":"2024-08-16T19:08:59.458335+0000","last_fullsized":"2024-08-16T19:08:59.458335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806708+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:08:14.806708+0000","last_peered":"2024-08-16T19:08:14.806708+0000","last_clean":"2024-08-16T19:08:14.806708+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:08:14.806708+0000","last_undegraded":"2024-08-16T19:08:14.806708+0000","last_fullsized":"2024-08-16T19:08:14.806708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455018+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:08:59.455018+0000","last_peered":"2024-08-16T19:08:59.455018+0000","last_clean":"2024-08-16T19:08:59.455018+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:08:59.455018+0000","last_undegraded":"2024-08-16T19:08:59.455018+0000","last_fullsized":"2024-08-16T19:08:59.455018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.805768+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:08:14.805768+0000","last_peered":"2024-08-16T19:08:14.805768+0000","last_clean":"2024-08-16T19:08:14.805768+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:08:14.805768+0000","last_undegraded":"2024-08-16T19:08:14.805768+0000","last_fullsized":"2024-08-16T19:08:14.805768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454239+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:08:59.454239+0000","last_peered":"2024-08-16T19:08:59.454239+0000","last_clean":"2024-08-16T19:08:59.454239+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:08:59.454239+0000","last_undegraded":"2024-08-16T19:08:59.454239+0000","last_fullsized":"2024-08-16T19:08:59.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:56.426502+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:08:56.426502+0000","last_peered":"2024-08-16T19:08:56.426502+0000","last_clean":"2024-08-16T19:08:56.426502+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:08:56.426502+0000","last_undegraded":"2024-08-16T19:08:56.426502+0000","last_fullsized":"2024-08-16T19:08:56.426502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806122+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:08:14.806122+0000","last_peered":"2024-08-16T19:08:14.806122+0000","last_clean":"2024-08-16T19:08:14.806122+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:08:14.806122+0000","last_undegraded":"2024-08-16T19:08:14.806122+0000","last_fullsized":"2024-08-16T19:08:14.806122+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.805267+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:08:14.805267+0000","last_peered":"2024-08-16T19:08:14.805267+0000","last_clean":"2024-08-16T19:08:14.805267+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:08:14.805267+0000","last_undegraded":"2024-08-16T19:08:14.805267+0000","last_fullsized":"2024-08-16T19:08:14.805267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454875+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:08:59.454875+0000","last_peered":"2024-08-16T19:08:59.454875+0000","last_clean":"2024-08-16T19:08:59.454875+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:08:59.454875+0000","last_undegraded":"2024-08-16T19:08:59.454875+0000","last_fullsized":"2024-08-16T19:08:59.454875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454959+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:08:59.454959+0000","last_peered":"2024-08-16T19:08:59.454959+0000","last_clean":"2024-08-16T19:08:59.454959+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:08:59.454959+0000","last_undegraded":"2024-08-16T19:08:59.454959+0000","last_fullsized":"2024-08-16T19:08:59.454959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.460782+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:08:58.460782+0000","last_peered":"2024-08-16T19:08:58.460782+0000","last_clean":"2024-08-16T19:08:58.460782+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:08:58.460782+0000","last_undegraded":"2024-08-16T19:08:58.460782+0000","last_fullsized":"2024-08-16T19:08:58.460782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458401+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:08:59.458401+0000","last_peered":"2024-08-16T19:08:59.458401+0000","last_clean":"2024-08-16T19:08:59.458401+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:08:59.458401+0000","last_undegraded":"2024-08-16T19:08:59.458401+0000","last_fullsized":"2024-08-16T19:08:59.458401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455187+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:08:59.455187+0000","last_peered":"2024-08-16T19:08:59.455187+0000","last_clean":"2024-08-16T19:08:59.455187+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:08:59.455187+0000","last_undegraded":"2024-08-16T19:08:59.455187+0000","last_fullsized":"2024-08-16T19:08:59.455187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455400+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:08:59.455400+0000","last_peered":"2024-08-16T19:08:59.455400+0000","last_clean":"2024-08-16T19:08:59.455400+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:08:59.455400+0000","last_undegraded":"2024-08-16T19:08:59.455400+0000","last_fullsized":"2024-08-16T19:08:59.455400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455750+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:08:59.455750+0000","last_peered":"2024-08-16T19:08:59.455750+0000","last_clean":"2024-08-16T19:08:59.455750+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:08:59.455750+0000","last_undegraded":"2024-08-16T19:08:59.455750+0000","last_fullsized":"2024-08-16T19:08:59.455750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:55.442809+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:08:55.442809+0000","last_peered":"2024-08-16T19:08:55.442809+0000","last_clean":"2024-08-16T19:08:55.442809+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:08:55.442809+0000","last_undegraded":"2024-08-16T19:08:55.442809+0000","last_fullsized":"2024-08-16T19:08:55.442809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545422+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:08:59.545422+0000","last_peered":"2024-08-16T19:08:59.545422+0000","last_clean":"2024-08-16T19:08:59.545422+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:08:59.545422+0000","last_undegraded":"2024-08-16T19:08:59.545422+0000","last_fullsized":"2024-08-16T19:08:59.545422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455331+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:08:59.455331+0000","last_peered":"2024-08-16T19:08:59.455331+0000","last_clean":"2024-08-16T19:08:59.455331+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:08:59.455331+0000","last_undegraded":"2024-08-16T19:08:59.455331+0000","last_fullsized":"2024-08-16T19:08:59.455331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454825+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:08:59.454825+0000","last_peered":"2024-08-16T19:08:59.454825+0000","last_clean":"2024-08-16T19:08:59.454825+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:08:59.454825+0000","last_undegraded":"2024-08-16T19:08:59.454825+0000","last_fullsized":"2024-08-16T19:08:59.454825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457861+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:08:59.457861+0000","last_peered":"2024-08-16T19:08:59.457861+0000","last_clean":"2024-08-16T19:08:59.457861+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:08:59.457861+0000","last_undegraded":"2024-08-16T19:08:59.457861+0000","last_fullsized":"2024-08-16T19:08:59.457861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455492+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:08:59.455492+0000","last_peered":"2024-08-16T19:08:59.455492+0000","last_clean":"2024-08-16T19:08:59.455492+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:08:59.455492+0000","last_undegraded":"2024-08-16T19:08:59.455492+0000","last_fullsized":"2024-08-16T19:08:59.455492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:47.421178+0000","last_change":"2024-08-16T19:08:02.457345+0000","last_active":"2024-08-16T19:08:47.421178+0000","last_peered":"2024-08-16T19:08:47.421178+0000","last_clean":"2024-08-16T19:08:47.421178+0000","last_became_active":"2024-08-16T19:08:02.456961+0000","last_became_peered":"2024-08-16T19:08:02.456961+0000","last_unstale":"2024-08-16T19:08:47.421178+0000","last_undegraded":"2024-08-16T19:08:47.421178+0000","last_fullsized":"2024-08-16T19:08:47.421178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:24.143886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454067+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:08:59.454067+0000","last_peered":"2024-08-16T19:08:59.454067+0000","last_clean":"2024-08-16T19:08:59.454067+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:08:59.454067+0000","last_undegraded":"2024-08-16T19:08:59.454067+0000","last_fullsized":"2024-08-16T19:08:59.454067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462353+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:08:59.462353+0000","last_peered":"2024-08-16T19:08:59.462353+0000","last_clean":"2024-08-16T19:08:59.462353+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:08:59.462353+0000","last_undegraded":"2024-08-16T19:08:59.462353+0000","last_fullsized":"2024-08-16T19:08:59.462353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459864+0000","last_change":"2024-08-16T19:08:02.457999+0000","last_active":"2024-08-16T19:08:59.459864+0000","last_peered":"2024-08-16T19:08:59.459864+0000","last_clean":"2024-08-16T19:08:59.459864+0000","last_became_active":"2024-08-16T19:08:02.456740+0000","last_became_peered":"2024-08-16T19:08:02.456740+0000","last_unstale":"2024-08-16T19:08:59.459864+0000","last_undegraded":"2024-08-16T19:08:59.459864+0000","last_fullsized":"2024-08-16T19:08:59.459864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:20:22.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454151+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:08:59.454151+0000","last_peered":"2024-08-16T19:08:59.454151+0000","last_clean":"2024-08-16T19:08:59.454151+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:08:59.454151+0000","last_undegraded":"2024-08-16T19:08:59.454151+0000","last_fullsized":"2024-08-16T19:08:59.454151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454277+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:08:59.454277+0000","last_peered":"2024-08-16T19:08:59.454277+0000","last_clean":"2024-08-16T19:08:59.454277+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:08:59.454277+0000","last_undegraded":"2024-08-16T19:08:59.454277+0000","last_fullsized":"2024-08-16T19:08:59.454277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458557+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:08:59.458557+0000","last_peered":"2024-08-16T19:08:59.458557+0000","last_clean":"2024-08-16T19:08:59.458557+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:08:59.458557+0000","last_undegraded":"2024-08-16T19:08:59.458557+0000","last_fullsized":"2024-08-16T19:08:59.458557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.805808+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:08:14.805808+0000","last_peered":"2024-08-16T19:08:14.805808+0000","last_clean":"2024-08-16T19:08:14.805808+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:08:14.805808+0000","last_undegraded":"2024-08-16T19:08:14.805808+0000","last_fullsized":"2024-08-16T19:08:14.805808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454634+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:08:59.454634+0000","last_peered":"2024-08-16T19:08:59.454634+0000","last_clean":"2024-08-16T19:08:59.454634+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:08:59.454634+0000","last_undegraded":"2024-08-16T19:08:59.454634+0000","last_fullsized":"2024-08-16T19:08:59.454634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458055+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:08:59.458055+0000","last_peered":"2024-08-16T19:08:59.458055+0000","last_clean":"2024-08-16T19:08:59.458055+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:08:59.458055+0000","last_undegraded":"2024-08-16T19:08:59.458055+0000","last_fullsized":"2024-08-16T19:08:59.458055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595244+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:08:59.595244+0000","last_peered":"2024-08-16T19:08:59.595244+0000","last_clean":"2024-08-16T19:08:59.595244+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:08:59.595244+0000","last_undegraded":"2024-08-16T19:08:59.595244+0000","last_fullsized":"2024-08-16T19:08:59.595244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455402+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:08:59.455402+0000","last_peered":"2024-08-16T19:08:59.455402+0000","last_clean":"2024-08-16T19:08:59.455402+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:08:59.455402+0000","last_undegraded":"2024-08-16T19:08:59.455402+0000","last_fullsized":"2024-08-16T19:08:59.455402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454397+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:08:59.454397+0000","last_peered":"2024-08-16T19:08:59.454397+0000","last_clean":"2024-08-16T19:08:59.454397+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:08:59.454397+0000","last_undegraded":"2024-08-16T19:08:59.454397+0000","last_fullsized":"2024-08-16T19:08:59.454397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462052+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:08:59.462052+0000","last_peered":"2024-08-16T19:08:59.462052+0000","last_clean":"2024-08-16T19:08:59.462052+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:08:59.462052+0000","last_undegraded":"2024-08-16T19:08:59.462052+0000","last_fullsized":"2024-08-16T19:08:59.462052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806039+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:08:14.806039+0000","last_peered":"2024-08-16T19:08:14.806039+0000","last_clean":"2024-08-16T19:08:14.806039+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:08:14.806039+0000","last_undegraded":"2024-08-16T19:08:14.806039+0000","last_fullsized":"2024-08-16T19:08:14.806039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:47.422603+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:08:47.422603+0000","last_peered":"2024-08-16T19:08:47.422603+0000","last_clean":"2024-08-16T19:08:47.422603+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:08:47.422603+0000","last_undegraded":"2024-08-16T19:08:47.422603+0000","last_fullsized":"2024-08-16T19:08:47.422603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.807218+0000","last_change":"2024-08-16T19:08:11.624941+0000","last_active":"2024-08-16T19:08:14.807218+0000","last_peered":"2024-08-16T19:08:14.807218+0000","last_clean":"2024-08-16T19:08:14.807218+0000","last_became_active":"2024-08-16T19:08:11.623978+0000","last_became_peered":"2024-08-16T19:08:11.623978+0000","last_unstale":"2024-08-16T19:08:14.807218+0000","last_undegraded":"2024-08-16T19:08:14.807218+0000","last_fullsized":"2024-08-16T19:08:14.807218+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:15:27.470156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459318+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:08:59.459318+0000","last_peered":"2024-08-16T19:08:59.459318+0000","last_clean":"2024-08-16T19:08:59.459318+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:08:59.459318+0000","last_undegraded":"2024-08-16T19:08:59.459318+0000","last_fullsized":"2024-08-16T19:08:59.459318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'27","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:56.434893+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:08:56.434893+0000","last_peered":"2024-08-16T19:08:56.434893+0000","last_clean":"2024-08-16T19:08:56.434893+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:08:56.434893+0000","last_undegraded":"2024-08-16T19:08:56.434893+0000","last_fullsized":"2024-08-16T19:08:56.434893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43339,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":56,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456560+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:08:59.456560+0000","last_peered":"2024-08-16T19:08:59.456560+0000","last_clean":"2024-08-16T19:08:59.456560+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:08:59.456560+0000","last_undegraded":"2024-08-16T19:08:59.456560+0000","last_fullsized":"2024-08-16T19:08:59.456560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546106+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:08:59.546106+0000","last_peered":"2024-08-16T19:08:59.546106+0000","last_clean":"2024-08-16T19:08:59.546106+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:08:59.546106+0000","last_undegraded":"2024-08-16T19:08:59.546106+0000","last_fullsized":"2024-08-16T19:08:59.546106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454804+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:08:59.454804+0000","last_peered":"2024-08-16T19:08:59.454804+0000","last_clean":"2024-08-16T19:08:59.454804+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:08:59.454804+0000","last_undegraded":"2024-08-16T19:08:59.454804+0000","last_fullsized":"2024-08-16T19:08:59.454804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458095+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:08:59.458095+0000","last_peered":"2024-08-16T19:08:59.458095+0000","last_clean":"2024-08-16T19:08:59.458095+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:08:59.458095+0000","last_undegraded":"2024-08-16T19:08:59.458095+0000","last_fullsized":"2024-08-16T19:08:59.458095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.460897+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:08:59.460897+0000","last_peered":"2024-08-16T19:08:59.460897+0000","last_clean":"2024-08-16T19:08:59.460897+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:08:59.460897+0000","last_undegraded":"2024-08-16T19:08:59.460897+0000","last_fullsized":"2024-08-16T19:08:59.460897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455542+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:08:59.455542+0000","last_peered":"2024-08-16T19:08:59.455542+0000","last_clean":"2024-08-16T19:08:59.455542+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:08:59.455542+0000","last_undegraded":"2024-08-16T19:08:59.455542+0000","last_fullsized":"2024-08-16T19:08:59.455542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456325+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:08:59.456325+0000","last_peered":"2024-08-16T19:08:59.456325+0000","last_clean":"2024-08-16T19:08:59.456325+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:08:59.456325+0000","last_undegraded":"2024-08-16T19:08:59.456325+0000","last_fullsized":"2024-08-16T19:08:59.456325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545951+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:08:59.545951+0000","last_peered":"2024-08-16T19:08:59.545951+0000","last_clean":"2024-08-16T19:08:59.545951+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:08:59.545951+0000","last_undegraded":"2024-08-16T19:08:59.545951+0000","last_fullsized":"2024-08-16T19:08:59.545951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:56.423432+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:08:56.423432+0000","last_peered":"2024-08-16T19:08:56.423432+0000","last_clean":"2024-08-16T19:08:56.423432+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:08:56.423432+0000","last_undegraded":"2024-08-16T19:08:56.423432+0000","last_fullsized":"2024-08-16T19:08:56.423432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.460111+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:08:58.460111+0000","last_peered":"2024-08-16T19:08:58.460111+0000","last_clean":"2024-08-16T19:08:58.460111+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:08:58.460111+0000","last_undegraded":"2024-08-16T19:08:58.460111+0000","last_fullsized":"2024-08-16T19:08:58.460111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457785+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:08:59.457785+0000","last_peered":"2024-08-16T19:08:59.457785+0000","last_clean":"2024-08-16T19:08:59.457785+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:08:59.457785+0000","last_undegraded":"2024-08-16T19:08:59.457785+0000","last_fullsized":"2024-08-16T19:08:59.457785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456321+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:08:59.456321+0000","last_peered":"2024-08-16T19:08:59.456321+0000","last_clean":"2024-08-16T19:08:59.456321+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:08:59.456321+0000","last_undegraded":"2024-08-16T19:08:59.456321+0000","last_fullsized":"2024-08-16T19:08:59.456321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454450+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:08:59.454450+0000","last_peered":"2024-08-16T19:08:59.454450+0000","last_clean":"2024-08-16T19:08:59.454450+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:08:59.454450+0000","last_undegraded":"2024-08-16T19:08:59.454450+0000","last_fullsized":"2024-08-16T19:08:59.454450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462131+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:08:59.462131+0000","last_peered":"2024-08-16T19:08:59.462131+0000","last_clean":"2024-08-16T19:08:59.462131+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:08:59.462131+0000","last_undegraded":"2024-08-16T19:08:59.462131+0000","last_fullsized":"2024-08-16T19:08:59.462131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.461530+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:08:58.461530+0000","last_peered":"2024-08-16T19:08:58.461530+0000","last_clean":"2024-08-16T19:08:58.461530+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:08:58.461530+0000","last_undegraded":"2024-08-16T19:08:58.461530+0000","last_fullsized":"2024-08-16T19:08:58.461530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:45.745521+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:08:45.745521+0000","last_peered":"2024-08-16T19:08:45.745521+0000","last_clean":"2024-08-16T19:08:45.745521+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:08:45.745521+0000","last_undegraded":"2024-08-16T19:08:45.745521+0000","last_fullsized":"2024-08-16T19:08:45.745521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:46.417218+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:08:46.417218+0000","last_peered":"2024-08-16T19:08:46.417218+0000","last_clean":"2024-08-16T19:08:46.417218+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:08:46.417218+0000","last_undegraded":"2024-08-16T19:08:46.417218+0000","last_fullsized":"2024-08-16T19:08:46.417218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454562+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:08:59.454562+0000","last_peered":"2024-08-16T19:08:59.454562+0000","last_clean":"2024-08-16T19:08:59.454562+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:08:59.454562+0000","last_undegraded":"2024-08-16T19:08:59.454562+0000","last_fullsized":"2024-08-16T19:08:59.454562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806199+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:08:14.806199+0000","last_peered":"2024-08-16T19:08:14.806199+0000","last_clean":"2024-08-16T19:08:14.806199+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:08:14.806199+0000","last_undegraded":"2024-08-16T19:08:14.806199+0000","last_fullsized":"2024-08-16T19:08:14.806199+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454682+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:08:59.454682+0000","last_peered":"2024-08-16T19:08:59.454682+0000","last_clean":"2024-08-16T19:08:59.454682+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:08:59.454682+0000","last_undegraded":"2024-08-16T19:08:59.454682+0000","last_fullsized":"2024-08-16T19:08:59.454682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458506+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:08:59.458506+0000","last_peered":"2024-08-16T19:08:59.458506+0000","last_clean":"2024-08-16T19:08:59.458506+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:08:59.458506+0000","last_undegraded":"2024-08-16T19:08:59.458506+0000","last_fullsized":"2024-08-16T19:08:59.458506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806501+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:08:14.806501+0000","last_peered":"2024-08-16T19:08:14.806501+0000","last_clean":"2024-08-16T19:08:14.806501+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:08:14.806501+0000","last_undegraded":"2024-08-16T19:08:14.806501+0000","last_fullsized":"2024-08-16T19:08:14.806501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462530+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:08:59.462530+0000","last_peered":"2024-08-16T19:08:59.462530+0000","last_clean":"2024-08-16T19:08:59.462530+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:08:59.462530+0000","last_undegraded":"2024-08-16T19:08:59.462530+0000","last_fullsized":"2024-08-16T19:08:59.462530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806915+0000","last_change":"2024-08-16T19:08:04.597250+0000","last_active":"2024-08-16T19:08:14.806915+0000","last_peered":"2024-08-16T19:08:14.806915+0000","last_clean":"2024-08-16T19:08:14.806915+0000","last_became_active":"2024-08-16T19:08:04.596916+0000","last_became_peered":"2024-08-16T19:08:04.596916+0000","last_unstale":"2024-08-16T19:08:14.806915+0000","last_undegraded":"2024-08-16T19:08:14.806915+0000","last_fullsized":"2024-08-16T19:08:14.806915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:27:07.322102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806123+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:08:14.806123+0000","last_peered":"2024-08-16T19:08:14.806123+0000","last_clean":"2024-08-16T19:08:14.806123+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:08:14.806123+0000","last_undegraded":"2024-08-16T19:08:14.806123+0000","last_fullsized":"2024-08-16T19:08:14.806123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457966+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:08:59.457966+0000","last_peered":"2024-08-16T19:08:59.457966+0000","last_clean":"2024-08-16T19:08:59.457966+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:08:59.457966+0000","last_undegraded":"2024-08-16T19:08:59.457966+0000","last_fullsized":"2024-08-16T19:08:59.457966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806071+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:08:14.806071+0000","last_peered":"2024-08-16T19:08:14.806071+0000","last_clean":"2024-08-16T19:08:14.806071+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:08:14.806071+0000","last_undegraded":"2024-08-16T19:08:14.806071+0000","last_fullsized":"2024-08-16T19:08:14.806071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462582+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:08:59.462582+0000","last_peered":"2024-08-16T19:08:59.462582+0000","last_clean":"2024-08-16T19:08:59.462582+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:08:59.462582+0000","last_undegraded":"2024-08-16T19:08:59.462582+0000","last_fullsized":"2024-08-16T19:08:59.462582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806613+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:08:14.806613+0000","last_peered":"2024-08-16T19:08:14.806613+0000","last_clean":"2024-08-16T19:08:14.806613+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:08:14.806613+0000","last_undegraded":"2024-08-16T19:08:14.806613+0000","last_fullsized":"2024-08-16T19:08:14.806613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.462281+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:08:58.462281+0000","last_peered":"2024-08-16T19:08:58.462281+0000","last_clean":"2024-08-16T19:08:58.462281+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:08:58.462281+0000","last_undegraded":"2024-08-16T19:08:58.462281+0000","last_fullsized":"2024-08-16T19:08:58.462281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595085+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:08:59.595085+0000","last_peered":"2024-08-16T19:08:59.595085+0000","last_clean":"2024-08-16T19:08:59.595085+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:08:59.595085+0000","last_undegraded":"2024-08-16T19:08:59.595085+0000","last_fullsized":"2024-08-16T19:08:59.595085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.460269+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:08:58.460269+0000","last_peered":"2024-08-16T19:08:58.460269+0000","last_clean":"2024-08-16T19:08:58.460269+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:08:58.460269+0000","last_undegraded":"2024-08-16T19:08:58.460269+0000","last_fullsized":"2024-08-16T19:08:58.460269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545609+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:08:59.545609+0000","last_peered":"2024-08-16T19:08:59.545609+0000","last_clean":"2024-08-16T19:08:59.545609+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:08:59.545609+0000","last_undegraded":"2024-08-16T19:08:59.545609+0000","last_fullsized":"2024-08-16T19:08:59.545609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.805915+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:08:14.805915+0000","last_peered":"2024-08-16T19:08:14.805915+0000","last_clean":"2024-08-16T19:08:14.805915+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:08:14.805915+0000","last_undegraded":"2024-08-16T19:08:14.805915+0000","last_fullsized":"2024-08-16T19:08:14.805915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466874+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:08:59.466874+0000","last_peered":"2024-08-16T19:08:59.466874+0000","last_clean":"2024-08-16T19:08:59.466874+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:08:59.466874+0000","last_undegraded":"2024-08-16T19:08:59.466874+0000","last_fullsized":"2024-08-16T19:08:59.466874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806545+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:08:14.806545+0000","last_peered":"2024-08-16T19:08:14.806545+0000","last_clean":"2024-08-16T19:08:14.806545+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:08:14.806545+0000","last_undegraded":"2024-08-16T19:08:14.806545+0000","last_fullsized":"2024-08-16T19:08:14.806545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462773+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:08:59.462773+0000","last_peered":"2024-08-16T19:08:59.462773+0000","last_clean":"2024-08-16T19:08:59.462773+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:08:59.462773+0000","last_undegraded":"2024-08-16T19:08:59.462773+0000","last_fullsized":"2024-08-16T19:08:59.462773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545175+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:08:59.545175+0000","last_peered":"2024-08-16T19:08:59.545175+0000","last_clean":"2024-08-16T19:08:59.545175+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:08:59.545175+0000","last_undegraded":"2024-08-16T19:08:59.545175+0000","last_fullsized":"2024-08-16T19:08:59.545175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457436+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:08:59.457436+0000","last_peered":"2024-08-16T19:08:59.457436+0000","last_clean":"2024-08-16T19:08:59.457436+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:08:59.457436+0000","last_undegraded":"2024-08-16T19:08:59.457436+0000","last_fullsized":"2024-08-16T19:08:59.457436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457698+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:08:59.457698+0000","last_peered":"2024-08-16T19:08:59.457698+0000","last_clean":"2024-08-16T19:08:59.457698+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:08:59.457698+0000","last_undegraded":"2024-08-16T19:08:59.457698+0000","last_fullsized":"2024-08-16T19:08:59.457698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.805738+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:08:14.805738+0000","last_peered":"2024-08-16T19:08:14.805738+0000","last_clean":"2024-08-16T19:08:14.805738+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:08:14.805738+0000","last_undegraded":"2024-08-16T19:08:14.805738+0000","last_fullsized":"2024-08-16T19:08:14.805738+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457704+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:08:59.457704+0000","last_peered":"2024-08-16T19:08:59.457704+0000","last_clean":"2024-08-16T19:08:59.457704+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:08:59.457704+0000","last_undegraded":"2024-08-16T19:08:59.457704+0000","last_fullsized":"2024-08-16T19:08:59.457704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455744+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:08:59.455744+0000","last_peered":"2024-08-16T19:08:59.455744+0000","last_clean":"2024-08-16T19:08:59.455744+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:08:59.455744+0000","last_undegraded":"2024-08-16T19:08:59.455744+0000","last_fullsized":"2024-08-16T19:08:59.455744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.807119+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:08:14.807119+0000","last_peered":"2024-08-16T19:08:14.807119+0000","last_clean":"2024-08-16T19:08:14.807119+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:08:14.807119+0000","last_undegraded":"2024-08-16T19:08:14.807119+0000","last_fullsized":"2024-08-16T19:08:14.807119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546015+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:08:59.546015+0000","last_peered":"2024-08-16T19:08:59.546015+0000","last_clean":"2024-08-16T19:08:59.546015+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:08:59.546015+0000","last_undegraded":"2024-08-16T19:08:59.546015+0000","last_fullsized":"2024-08-16T19:08:59.546015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-08-16T19:08:58.461731+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:08:58.461731+0000","last_peered":"2024-08-16T19:08:58.461731+0000","last_clean":"2024-08-16T19:08:58.461731+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:08:58.461731+0000","last_undegraded":"2024-08-16T19:08:58.461731+0000","last_fullsized":"2024-08-16T19:08:58.461731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454659+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:08:59.454659+0000","last_peered":"2024-08-16T19:08:59.454659+0000","last_clean":"2024-08-16T19:08:59.454659+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:08:59.454659+0000","last_undegraded":"2024-08-16T19:08:59.454659+0000","last_fullsized":"2024-08-16T19:08:59.454659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-16T19:08:14.806509+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:08:14.806509+0000","last_peered":"2024-08-16T19:08:14.806509+0000","last_clean":"2024-08-16T19:08:14.806509+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:08:14.806509+0000","last_undegraded":"2024-08-16T19:08:14.806509+0000","last_fullsized":"2024-08-16T19:08:14.806509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458460+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:08:59.458460+0000","last_peered":"2024-08-16T19:08:59.458460+0000","last_clean":"2024-08-16T19:08:59.458460+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:08:59.458460+0000","last_undegraded":"2024-08-16T19:08:59.458460+0000","last_fullsized":"2024-08-16T19:08:59.458460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457358+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:08:59.457358+0000","last_peered":"2024-08-16T19:08:59.457358+0000","last_clean":"2024-08-16T19:08:59.457358+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:08:59.457358+0000","last_undegraded":"2024-08-16T19:08:59.457358+0000","last_fullsized":"2024-08-16T19:08:59.457358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458036+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:08:59.458036+0000","last_peered":"2024-08-16T19:08:59.458036+0000","last_clean":"2024-08-16T19:08:59.458036+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:08:59.458036+0000","last_undegraded":"2024-08-16T19:08:59.458036+0000","last_fullsized":"2024-08-16T19:08:59.458036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457901+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:08:59.457901+0000","last_peered":"2024-08-16T19:08:59.457901+0000","last_clean":"2024-08-16T19:08:59.457901+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:08:59.457901+0000","last_undegraded":"2024-08-16T19:08:59.457901+0000","last_fullsized":"2024-08-16T19:08:59.457901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462449+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:08:59.462449+0000","last_peered":"2024-08-16T19:08:59.462449+0000","last_clean":"2024-08-16T19:08:59.462449+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:08:59.462449+0000","last_undegraded":"2024-08-16T19:08:59.462449+0000","last_fullsized":"2024-08-16T19:08:59.462449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50931,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":154,"num_read_kb":133,"num_write":238,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":166121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1120225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":512644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":508926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430250,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1105812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593774,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":485504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822707,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1100614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":508516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":484822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":530093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":562139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":505898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:09:03.010 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:02 smithi062 ceph-mon[32993]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:02 smithi155 ceph-mon[37413]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:03.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-08-16T19:09:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:03.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:02 smithi161 ceph-mon[37576]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:03.287 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:03.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 -- 172.21.15.62:0/3536098313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd30103390 msgr2=0x7fcd3010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 --2- 172.21.15.62:0/3536098313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd30103390 0x7fcd3010b880 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009900 tx=0x7fcd2402a480 comp rx=0 tx=0).stop 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 -- 172.21.15.62:0/3536098313 shutdown_connections 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 --2- 172.21.15.62:0/3536098313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd3010bea0 0x7fcd3010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 --2- 172.21.15.62:0/3536098313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd30103390 0x7fcd3010b880 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 --2- 172.21.15.62:0/3536098313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 0x7fcd30102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 -- 172.21.15.62:0/3536098313 >> 172.21.15.62:0/3536098313 conn(0x7fcd300fc4d0 msgr2=0x7fcd300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 -- 172.21.15.62:0/3536098313 shutdown_connections 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.779+0000 7fcd35e94640 1 -- 172.21.15.62:0/3536098313 wait complete. 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.780+0000 7fcd35e94640 1 Processor -- start 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.780+0000 7fcd35e94640 1 -- start start 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 0x7fcd301006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:03.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd30103390 0x7fcd30100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30111d30 con 0x7fcd3010bea0 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30111bb0 con 0x7fcd30103390 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd35e94640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd30111eb0 con 0x7fcd30102a70 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50434/0 (socket says 172.21.15.62:50434) 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 -- 172.21.15.62:0/1233519894 learned_addr learned my addr 172.21.15.62:0/1233519894 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2effd640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd30103390 0x7fcd30100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2f7fe640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 0x7fcd301006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 -- 172.21.15.62:0/1233519894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 msgr2=0x7fcd301006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 0x7fcd301006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 -- 172.21.15.62:0/1233519894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd30103390 msgr2=0x7fcd30100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:03.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd30103390 0x7fcd30100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 -- 172.21.15.62:0/1233519894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd301a6650 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2ffff640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800cce0 tx=0x7fcd2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800f450 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2800fad0 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.781+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28018c10 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.782+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301a6940 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.782+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301a6e00 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.783+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd30102e50 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.784+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcd2800f5f0 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.784+0000 7fcd2cff9640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 0x7fcd180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.784+0000 7fcd2f7fe640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 0x7fcd180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.784+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fcd280cfc20 con 0x7fcd3010bea0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.785+0000 7fcd2f7fe640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 0x7fcd180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009890 tx=0x7fcd20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:03.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.788+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcd28014030 con 0x7fcd3010bea0 2024-08-16T19:09:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.961+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd300fe7f0 con 0x7fcd180a2c10 2024-08-16T19:09:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.969+0000 7fcd2cff9640 1 -- 172.21.15.62:0/1233519894 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340438 (secure 0 0 0) 0x7fcd300fe7f0 con 0x7fcd180a2c10 2024-08-16T19:09:03.973 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:03.975 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 msgr2=0x7fcd180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 0x7fcd180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009890 tx=0x7fcd20008040 comp rx=0 tx=0).stop 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 msgr2=0x7fcd30101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800cce0 tx=0x7fcd2800a620 comp rx=0 tx=0).stop 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 shutdown_connections 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd3010bea0 0x7fcd30101150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fcd180a2c10 0x7fcd180a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd30103390 0x7fcd30100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 --2- 172.21.15.62:0/1233519894 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd30102a70 0x7fcd301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:03.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 >> 172.21.15.62:0/1233519894 conn(0x7fcd300fc4d0 msgr2=0x7fcd300fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:03.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 shutdown_connections 2024-08-16T19:09:03.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:03.973+0000 7fcd35e94640 1 -- 172.21.15.62:0/1233519894 wait complete. 2024-08-16T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:09:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:04 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:09:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:09:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:04 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:09:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:09:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:04 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:09:04.731 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-08-16T19:09:03.659881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654861,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":321,"num_read_kb":286,"num_write":425,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375932,"kb_used_data":14216,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565892,"statfs":{"total":1151940427776,"available":1151555473408,"internally_reserved":0,"allocated":14557184,"data_stored":8036609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32093,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":92,"num_write":16,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001881"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457998+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:08:59.457998+0000","last_peered":"2024-08-16T19:08:59.457998+0000","last_clean":"2024-08-16T19:08:59.457998+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:08:59.457998+0000","last_undegraded":"2024-08-16T19:08:59.457998+0000","last_fullsized":"2024-08-16T19:08:59.457998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592994+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:08:59.592994+0000","last_peered":"2024-08-16T19:08:59.592994+0000","last_clean":"2024-08-16T19:08:59.592994+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:08:59.592994+0000","last_undegraded":"2024-08-16T19:08:59.592994+0000","last_fullsized":"2024-08-16T19:08:59.592994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456279+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:08:59.456279+0000","last_peered":"2024-08-16T19:08:59.456279+0000","last_clean":"2024-08-16T19:08:59.456279+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:08:59.456279+0000","last_undegraded":"2024-08-16T19:08:59.456279+0000","last_fullsized":"2024-08-16T19:08:59.456279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593258+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:08:59.593258+0000","last_peered":"2024-08-16T19:08:59.593258+0000","last_clean":"2024-08-16T19:08:59.593258+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:08:59.593258+0000","last_undegraded":"2024-08-16T19:08:59.593258+0000","last_fullsized":"2024-08-16T19:08:59.593258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458615+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:08:59.458615+0000","last_peered":"2024-08-16T19:08:59.458615+0000","last_clean":"2024-08-16T19:08:59.458615+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:08:59.458615+0000","last_undegraded":"2024-08-16T19:08:59.458615+0000","last_fullsized":"2024-08-16T19:08:59.458615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.594556+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:08:59.594556+0000","last_peered":"2024-08-16T19:08:59.594556+0000","last_clean":"2024-08-16T19:08:59.594556+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:08:59.594556+0000","last_undegraded":"2024-08-16T19:08:59.594556+0000","last_fullsized":"2024-08-16T19:08:59.594556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459473+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:08:59.459473+0000","last_peered":"2024-08-16T19:08:59.459473+0000","last_clean":"2024-08-16T19:08:59.459473+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:08:59.459473+0000","last_undegraded":"2024-08-16T19:08:59.459473+0000","last_fullsized":"2024-08-16T19:08:59.459473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545800+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:08:59.545800+0000","last_peered":"2024-08-16T19:08:59.545800+0000","last_clean":"2024-08-16T19:08:59.545800+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:08:59.545800+0000","last_undegraded":"2024-08-16T19:08:59.545800+0000","last_fullsized":"2024-08-16T19:08:59.545800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457100+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:08:59.457100+0000","last_peered":"2024-08-16T19:08:59.457100+0000","last_clean":"2024-08-16T19:08:59.457100+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:08:59.457100+0000","last_undegraded":"2024-08-16T19:08:59.457100+0000","last_fullsized":"2024-08-16T19:08:59.457100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459443+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:08:59.459443+0000","last_peered":"2024-08-16T19:08:59.459443+0000","last_clean":"2024-08-16T19:08:59.459443+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:08:59.459443+0000","last_undegraded":"2024-08-16T19:08:59.459443+0000","last_fullsized":"2024-08-16T19:08:59.459443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.547859+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:08:59.547859+0000","last_peered":"2024-08-16T19:08:59.547859+0000","last_clean":"2024-08-16T19:08:59.547859+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:08:59.547859+0000","last_undegraded":"2024-08-16T19:08:59.547859+0000","last_fullsized":"2024-08-16T19:08:59.547859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457189+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:08:59.457189+0000","last_peered":"2024-08-16T19:08:59.457189+0000","last_clean":"2024-08-16T19:08:59.457189+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:08:59.457189+0000","last_undegraded":"2024-08-16T19:08:59.457189+0000","last_fullsized":"2024-08-16T19:08:59.457189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454902+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:08:59.454902+0000","last_peered":"2024-08-16T19:08:59.454902+0000","last_clean":"2024-08-16T19:08:59.454902+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:08:59.454902+0000","last_undegraded":"2024-08-16T19:08:59.454902+0000","last_fullsized":"2024-08-16T19:08:59.454902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458666+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:08:59.458666+0000","last_peered":"2024-08-16T19:08:59.458666+0000","last_clean":"2024-08-16T19:08:59.458666+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:08:59.458666+0000","last_undegraded":"2024-08-16T19:08:59.458666+0000","last_fullsized":"2024-08-16T19:08:59.458666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457164+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:08:59.457164+0000","last_peered":"2024-08-16T19:08:59.457164+0000","last_clean":"2024-08-16T19:08:59.457164+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:08:59.457164+0000","last_undegraded":"2024-08-16T19:08:59.457164+0000","last_fullsized":"2024-08-16T19:08:59.457164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592793+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:08:59.592793+0000","last_peered":"2024-08-16T19:08:59.592793+0000","last_clean":"2024-08-16T19:08:59.592793+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:08:59.592793+0000","last_undegraded":"2024-08-16T19:08:59.592793+0000","last_fullsized":"2024-08-16T19:08:59.592793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456614+0000","last_change":"2024-08-16T19:08:02.462372+0000","last_active":"2024-08-16T19:08:59.456614+0000","last_peered":"2024-08-16T19:08:59.456614+0000","last_clean":"2024-08-16T19:08:59.456614+0000","last_became_active":"2024-08-16T19:08:02.461973+0000","last_became_peered":"2024-08-16T19:08:02.461973+0000","last_unstale":"2024-08-16T19:08:59.456614+0000","last_undegraded":"2024-08-16T19:08:59.456614+0000","last_fullsized":"2024-08-16T19:08:59.456614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:24:04.191300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457135+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:08:59.457135+0000","last_peered":"2024-08-16T19:08:59.457135+0000","last_clean":"2024-08-16T19:08:59.457135+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:08:59.457135+0000","last_undegraded":"2024-08-16T19:08:59.457135+0000","last_fullsized":"2024-08-16T19:08:59.457135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462630+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:08:59.462630+0000","last_peered":"2024-08-16T19:08:59.462630+0000","last_clean":"2024-08-16T19:08:59.462630+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:08:59.462630+0000","last_undegraded":"2024-08-16T19:08:59.462630+0000","last_fullsized":"2024-08-16T19:08:59.462630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593234+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:08:59.593234+0000","last_peered":"2024-08-16T19:08:59.593234+0000","last_clean":"2024-08-16T19:08:59.593234+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:08:59.593234+0000","last_undegraded":"2024-08-16T19:08:59.593234+0000","last_fullsized":"2024-08-16T19:08:59.593234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457650+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:08:59.457650+0000","last_peered":"2024-08-16T19:08:59.457650+0000","last_clean":"2024-08-16T19:08:59.457650+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:08:59.457650+0000","last_undegraded":"2024-08-16T19:08:59.457650+0000","last_fullsized":"2024-08-16T19:08:59.457650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458319+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:08:59.458319+0000","last_peered":"2024-08-16T19:08:59.458319+0000","last_clean":"2024-08-16T19:08:59.458319+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:08:59.458319+0000","last_undegraded":"2024-08-16T19:08:59.458319+0000","last_fullsized":"2024-08-16T19:08:59.458319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545351+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:08:59.545351+0000","last_peered":"2024-08-16T19:08:59.545351+0000","last_clean":"2024-08-16T19:08:59.545351+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:08:59.545351+0000","last_undegraded":"2024-08-16T19:08:59.545351+0000","last_fullsized":"2024-08-16T19:08:59.545351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.464030+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:08:59.464030+0000","last_peered":"2024-08-16T19:08:59.464030+0000","last_clean":"2024-08-16T19:08:59.464030+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:08:59.464030+0000","last_undegraded":"2024-08-16T19:08:59.464030+0000","last_fullsized":"2024-08-16T19:08:59.464030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458222+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:08:59.458222+0000","last_peered":"2024-08-16T19:08:59.458222+0000","last_clean":"2024-08-16T19:08:59.458222+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:08:59.458222+0000","last_undegraded":"2024-08-16T19:08:59.458222+0000","last_fullsized":"2024-08-16T19:08:59.458222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458400+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:08:59.458400+0000","last_peered":"2024-08-16T19:08:59.458400+0000","last_clean":"2024-08-16T19:08:59.458400+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:08:59.458400+0000","last_undegraded":"2024-08-16T19:08:59.458400+0000","last_fullsized":"2024-08-16T19:08:59.458400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456408+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:08:59.456408+0000","last_peered":"2024-08-16T19:08:59.456408+0000","last_clean":"2024-08-16T19:08:59.456408+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:08:59.456408+0000","last_undegraded":"2024-08-16T19:08:59.456408+0000","last_fullsized":"2024-08-16T19:08:59.456408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466015+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:08:59.466015+0000","last_peered":"2024-08-16T19:08:59.466015+0000","last_clean":"2024-08-16T19:08:59.466015+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:08:59.466015+0000","last_undegraded":"2024-08-16T19:08:59.466015+0000","last_fullsized":"2024-08-16T19:08:59.466015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456637+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:08:59.456637+0000","last_peered":"2024-08-16T19:08:59.456637+0000","last_clean":"2024-08-16T19:08:59.456637+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:08:59.456637+0000","last_undegraded":"2024-08-16T19:08:59.456637+0000","last_fullsized":"2024-08-16T19:08:59.456637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593074+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:08:59.593074+0000","last_peered":"2024-08-16T19:08:59.593074+0000","last_clean":"2024-08-16T19:08:59.593074+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:08:59.593074+0000","last_undegraded":"2024-08-16T19:08:59.593074+0000","last_fullsized":"2024-08-16T19:08:59.593074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455577+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:08:59.455577+0000","last_peered":"2024-08-16T19:08:59.455577+0000","last_clean":"2024-08-16T19:08:59.455577+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:08:59.455577+0000","last_undegraded":"2024-08-16T19:08:59.455577+0000","last_fullsized":"2024-08-16T19:08:59.455577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459679+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:08:59.459679+0000","last_peered":"2024-08-16T19:08:59.459679+0000","last_clean":"2024-08-16T19:08:59.459679+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:08:59.459679+0000","last_undegraded":"2024-08-16T19:08:59.459679+0000","last_fullsized":"2024-08-16T19:08:59.459679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462351+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:08:59.462351+0000","last_peered":"2024-08-16T19:08:59.462351+0000","last_clean":"2024-08-16T19:08:59.462351+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:08:59.462351+0000","last_undegraded":"2024-08-16T19:08:59.462351+0000","last_fullsized":"2024-08-16T19:08:59.462351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458157+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:08:59.458157+0000","last_peered":"2024-08-16T19:08:59.458157+0000","last_clean":"2024-08-16T19:08:59.458157+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:08:59.458157+0000","last_undegraded":"2024-08-16T19:08:59.458157+0000","last_fullsized":"2024-08-16T19:08:59.458157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456378+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:08:59.456378+0000","last_peered":"2024-08-16T19:08:59.456378+0000","last_clean":"2024-08-16T19:08:59.456378+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:08:59.456378+0000","last_undegraded":"2024-08-16T19:08:59.456378+0000","last_fullsized":"2024-08-16T19:08:59.456378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458335+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:08:59.458335+0000","last_peered":"2024-08-16T19:08:59.458335+0000","last_clean":"2024-08-16T19:08:59.458335+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:08:59.458335+0000","last_undegraded":"2024-08-16T19:08:59.458335+0000","last_fullsized":"2024-08-16T19:08:59.458335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457766+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:08:59.457766+0000","last_peered":"2024-08-16T19:08:59.457766+0000","last_clean":"2024-08-16T19:08:59.457766+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:08:59.457766+0000","last_undegraded":"2024-08-16T19:08:59.457766+0000","last_fullsized":"2024-08-16T19:08:59.457766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455018+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:08:59.455018+0000","last_peered":"2024-08-16T19:08:59.455018+0000","last_clean":"2024-08-16T19:08:59.455018+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:08:59.455018+0000","last_undegraded":"2024-08-16T19:08:59.455018+0000","last_fullsized":"2024-08-16T19:08:59.455018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455413+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:08:59.455413+0000","last_peered":"2024-08-16T19:08:59.455413+0000","last_clean":"2024-08-16T19:08:59.455413+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:08:59.455413+0000","last_undegraded":"2024-08-16T19:08:59.455413+0000","last_fullsized":"2024-08-16T19:08:59.455413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454239+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:08:59.454239+0000","last_peered":"2024-08-16T19:08:59.454239+0000","last_clean":"2024-08-16T19:08:59.454239+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:08:59.454239+0000","last_undegraded":"2024-08-16T19:08:59.454239+0000","last_fullsized":"2024-08-16T19:08:59.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:01.432528+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:09:01.432528+0000","last_peered":"2024-08-16T19:09:01.432528+0000","last_clean":"2024-08-16T19:09:01.432528+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:09:01.432528+0000","last_undegraded":"2024-08-16T19:09:01.432528+0000","last_fullsized":"2024-08-16T19:09:01.432528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455963+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:08:59.455963+0000","last_peered":"2024-08-16T19:08:59.455963+0000","last_clean":"2024-08-16T19:08:59.455963+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:08:59.455963+0000","last_undegraded":"2024-08-16T19:08:59.455963+0000","last_fullsized":"2024-08-16T19:08:59.455963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455170+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:08:59.455170+0000","last_peered":"2024-08-16T19:08:59.455170+0000","last_clean":"2024-08-16T19:08:59.455170+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:08:59.455170+0000","last_undegraded":"2024-08-16T19:08:59.455170+0000","last_fullsized":"2024-08-16T19:08:59.455170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454875+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:08:59.454875+0000","last_peered":"2024-08-16T19:08:59.454875+0000","last_clean":"2024-08-16T19:08:59.454875+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:08:59.454875+0000","last_undegraded":"2024-08-16T19:08:59.454875+0000","last_fullsized":"2024-08-16T19:08:59.454875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454959+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:08:59.454959+0000","last_peered":"2024-08-16T19:08:59.454959+0000","last_clean":"2024-08-16T19:08:59.454959+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:08:59.454959+0000","last_undegraded":"2024-08-16T19:08:59.454959+0000","last_fullsized":"2024-08-16T19:08:59.454959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592928+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:08:59.592928+0000","last_peered":"2024-08-16T19:08:59.592928+0000","last_clean":"2024-08-16T19:08:59.592928+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:08:59.592928+0000","last_undegraded":"2024-08-16T19:08:59.592928+0000","last_fullsized":"2024-08-16T19:08:59.592928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458401+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:08:59.458401+0000","last_peered":"2024-08-16T19:08:59.458401+0000","last_clean":"2024-08-16T19:08:59.458401+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:08:59.458401+0000","last_undegraded":"2024-08-16T19:08:59.458401+0000","last_fullsized":"2024-08-16T19:08:59.458401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455187+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:08:59.455187+0000","last_peered":"2024-08-16T19:08:59.455187+0000","last_clean":"2024-08-16T19:08:59.455187+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:08:59.455187+0000","last_undegraded":"2024-08-16T19:08:59.455187+0000","last_fullsized":"2024-08-16T19:08:59.455187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455400+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:08:59.455400+0000","last_peered":"2024-08-16T19:08:59.455400+0000","last_clean":"2024-08-16T19:08:59.455400+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:08:59.455400+0000","last_undegraded":"2024-08-16T19:08:59.455400+0000","last_fullsized":"2024-08-16T19:08:59.455400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455750+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:08:59.455750+0000","last_peered":"2024-08-16T19:08:59.455750+0000","last_clean":"2024-08-16T19:08:59.455750+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:08:59.455750+0000","last_undegraded":"2024-08-16T19:08:59.455750+0000","last_fullsized":"2024-08-16T19:08:59.455750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:00.448881+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:09:00.448881+0000","last_peered":"2024-08-16T19:09:00.448881+0000","last_clean":"2024-08-16T19:09:00.448881+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:09:00.448881+0000","last_undegraded":"2024-08-16T19:09:00.448881+0000","last_fullsized":"2024-08-16T19:09:00.448881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545422+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:08:59.545422+0000","last_peered":"2024-08-16T19:08:59.545422+0000","last_clean":"2024-08-16T19:08:59.545422+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:08:59.545422+0000","last_undegraded":"2024-08-16T19:08:59.545422+0000","last_fullsized":"2024-08-16T19:08:59.545422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455331+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:08:59.455331+0000","last_peered":"2024-08-16T19:08:59.455331+0000","last_clean":"2024-08-16T19:08:59.455331+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:08:59.455331+0000","last_undegraded":"2024-08-16T19:08:59.455331+0000","last_fullsized":"2024-08-16T19:08:59.455331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454825+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:08:59.454825+0000","last_peered":"2024-08-16T19:08:59.454825+0000","last_clean":"2024-08-16T19:08:59.454825+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:08:59.454825+0000","last_undegraded":"2024-08-16T19:08:59.454825+0000","last_fullsized":"2024-08-16T19:08:59.454825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457861+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:08:59.457861+0000","last_peered":"2024-08-16T19:08:59.457861+0000","last_clean":"2024-08-16T19:08:59.457861+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:08:59.457861+0000","last_undegraded":"2024-08-16T19:08:59.457861+0000","last_fullsized":"2024-08-16T19:08:59.457861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455492+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:08:59.455492+0000","last_peered":"2024-08-16T19:08:59.455492+0000","last_clean":"2024-08-16T19:08:59.455492+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:08:59.455492+0000","last_undegraded":"2024-08-16T19:08:59.455492+0000","last_fullsized":"2024-08-16T19:08:59.455492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455979+0000","last_change":"2024-08-16T19:08:02.457345+0000","last_active":"2024-08-16T19:08:59.455979+0000","last_peered":"2024-08-16T19:08:59.455979+0000","last_clean":"2024-08-16T19:08:59.455979+0000","last_became_active":"2024-08-16T19:08:02.456961+0000","last_became_peered":"2024-08-16T19:08:02.456961+0000","last_unstale":"2024-08-16T19:08:59.455979+0000","last_undegraded":"2024-08-16T19:08:59.455979+0000","last_fullsized":"2024-08-16T19:08:59.455979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:24.143886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454067+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:08:59.454067+0000","last_peered":"2024-08-16T19:08:59.454067+0000","last_clean":"2024-08-16T19:08:59.454067+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:08:59.454067+0000","last_undegraded":"2024-08-16T19:08:59.454067+0000","last_fullsized":"2024-08-16T19:08:59.454067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462353+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:08:59.462353+0000","last_peered":"2024-08-16T19:08:59.462353+0000","last_clean":"2024-08-16T19:08:59.462353+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:08:59.462353+0000","last_undegraded":"2024-08-16T19:08:59.462353+0000","last_fullsized":"2024-08-16T19:08:59.462353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459864+0000","last_change":"2024-08-16T19:08:02.457999+0000","last_active":"2024-08-16T19:08:59.459864+0000","last_peered":"2024-08-16T19:08:59.459864+0000","last_clean":"2024-08-16T19:08:59.459864+0000","last_became_active":"2024-08-16T19:08:02.456740+0000","last_became_peered":"2024-08-16T19:08:02.456740+0000","last_unstale":"2024-08-16T19:08:59.459864+0000","last_undegraded":"2024-08-16T19:08:59.459864+0000","last_fullsized":"2024-08-16T19:08:59.459864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:20:22.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454151+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:08:59.454151+0000","last_peered":"2024-08-16T19:08:59.454151+0000","last_clean":"2024-08-16T19:08:59.454151+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:08:59.454151+0000","last_undegraded":"2024-08-16T19:08:59.454151+0000","last_fullsized":"2024-08-16T19:08:59.454151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454277+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:08:59.454277+0000","last_peered":"2024-08-16T19:08:59.454277+0000","last_clean":"2024-08-16T19:08:59.454277+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:08:59.454277+0000","last_undegraded":"2024-08-16T19:08:59.454277+0000","last_fullsized":"2024-08-16T19:08:59.454277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458557+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:08:59.458557+0000","last_peered":"2024-08-16T19:08:59.458557+0000","last_clean":"2024-08-16T19:08:59.458557+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:08:59.458557+0000","last_undegraded":"2024-08-16T19:08:59.458557+0000","last_fullsized":"2024-08-16T19:08:59.458557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455944+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:08:59.455944+0000","last_peered":"2024-08-16T19:08:59.455944+0000","last_clean":"2024-08-16T19:08:59.455944+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:08:59.455944+0000","last_undegraded":"2024-08-16T19:08:59.455944+0000","last_fullsized":"2024-08-16T19:08:59.455944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454634+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:08:59.454634+0000","last_peered":"2024-08-16T19:08:59.454634+0000","last_clean":"2024-08-16T19:08:59.454634+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:08:59.454634+0000","last_undegraded":"2024-08-16T19:08:59.454634+0000","last_fullsized":"2024-08-16T19:08:59.454634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458055+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:08:59.458055+0000","last_peered":"2024-08-16T19:08:59.458055+0000","last_clean":"2024-08-16T19:08:59.458055+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:08:59.458055+0000","last_undegraded":"2024-08-16T19:08:59.458055+0000","last_fullsized":"2024-08-16T19:08:59.458055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595244+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:08:59.595244+0000","last_peered":"2024-08-16T19:08:59.595244+0000","last_clean":"2024-08-16T19:08:59.595244+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:08:59.595244+0000","last_undegraded":"2024-08-16T19:08:59.595244+0000","last_fullsized":"2024-08-16T19:08:59.595244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455402+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:08:59.455402+0000","last_peered":"2024-08-16T19:08:59.455402+0000","last_clean":"2024-08-16T19:08:59.455402+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:08:59.455402+0000","last_undegraded":"2024-08-16T19:08:59.455402+0000","last_fullsized":"2024-08-16T19:08:59.455402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454397+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:08:59.454397+0000","last_peered":"2024-08-16T19:08:59.454397+0000","last_clean":"2024-08-16T19:08:59.454397+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:08:59.454397+0000","last_undegraded":"2024-08-16T19:08:59.454397+0000","last_fullsized":"2024-08-16T19:08:59.454397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462052+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:08:59.462052+0000","last_peered":"2024-08-16T19:08:59.462052+0000","last_clean":"2024-08-16T19:08:59.462052+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:08:59.462052+0000","last_undegraded":"2024-08-16T19:08:59.462052+0000","last_fullsized":"2024-08-16T19:08:59.462052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455923+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:08:59.455923+0000","last_peered":"2024-08-16T19:08:59.455923+0000","last_clean":"2024-08-16T19:08:59.455923+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:08:59.455923+0000","last_undegraded":"2024-08-16T19:08:59.455923+0000","last_fullsized":"2024-08-16T19:08:59.455923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595109+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:08:59.595109+0000","last_peered":"2024-08-16T19:08:59.595109+0000","last_clean":"2024-08-16T19:08:59.595109+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:08:59.595109+0000","last_undegraded":"2024-08-16T19:08:59.595109+0000","last_fullsized":"2024-08-16T19:08:59.595109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459318+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:08:59.459318+0000","last_peered":"2024-08-16T19:08:59.459318+0000","last_clean":"2024-08-16T19:08:59.459318+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:08:59.459318+0000","last_undegraded":"2024-08-16T19:08:59.459318+0000","last_fullsized":"2024-08-16T19:08:59.459318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":103,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:01.441724+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:09:01.441724+0000","last_peered":"2024-08-16T19:09:01.441724+0000","last_clean":"2024-08-16T19:09:01.441724+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:09:01.441724+0000","last_undegraded":"2024-08-16T19:09:01.441724+0000","last_fullsized":"2024-08-16T19:09:01.441724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56667,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":75,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456560+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:08:59.456560+0000","last_peered":"2024-08-16T19:08:59.456560+0000","last_clean":"2024-08-16T19:08:59.456560+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:08:59.456560+0000","last_undegraded":"2024-08-16T19:08:59.456560+0000","last_fullsized":"2024-08-16T19:08:59.456560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546106+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:08:59.546106+0000","last_peered":"2024-08-16T19:08:59.546106+0000","last_clean":"2024-08-16T19:08:59.546106+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:08:59.546106+0000","last_undegraded":"2024-08-16T19:08:59.546106+0000","last_fullsized":"2024-08-16T19:08:59.546106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454804+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:08:59.454804+0000","last_peered":"2024-08-16T19:08:59.454804+0000","last_clean":"2024-08-16T19:08:59.454804+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:08:59.454804+0000","last_undegraded":"2024-08-16T19:08:59.454804+0000","last_fullsized":"2024-08-16T19:08:59.454804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458095+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:08:59.458095+0000","last_peered":"2024-08-16T19:08:59.458095+0000","last_clean":"2024-08-16T19:08:59.458095+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:08:59.458095+0000","last_undegraded":"2024-08-16T19:08:59.458095+0000","last_fullsized":"2024-08-16T19:08:59.458095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.460897+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:08:59.460897+0000","last_peered":"2024-08-16T19:08:59.460897+0000","last_clean":"2024-08-16T19:08:59.460897+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:08:59.460897+0000","last_undegraded":"2024-08-16T19:08:59.460897+0000","last_fullsized":"2024-08-16T19:08:59.460897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455542+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:08:59.455542+0000","last_peered":"2024-08-16T19:08:59.455542+0000","last_clean":"2024-08-16T19:08:59.455542+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:08:59.455542+0000","last_undegraded":"2024-08-16T19:08:59.455542+0000","last_fullsized":"2024-08-16T19:08:59.455542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456325+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:08:59.456325+0000","last_peered":"2024-08-16T19:08:59.456325+0000","last_clean":"2024-08-16T19:08:59.456325+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:08:59.456325+0000","last_undegraded":"2024-08-16T19:08:59.456325+0000","last_fullsized":"2024-08-16T19:08:59.456325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545951+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:08:59.545951+0000","last_peered":"2024-08-16T19:08:59.545951+0000","last_clean":"2024-08-16T19:08:59.545951+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:08:59.545951+0000","last_undegraded":"2024-08-16T19:08:59.545951+0000","last_fullsized":"2024-08-16T19:08:59.545951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:01.431521+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:09:01.431521+0000","last_peered":"2024-08-16T19:09:01.431521+0000","last_clean":"2024-08-16T19:09:01.431521+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:09:01.431521+0000","last_undegraded":"2024-08-16T19:09:01.431521+0000","last_fullsized":"2024-08-16T19:09:01.431521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592679+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:08:59.592679+0000","last_peered":"2024-08-16T19:08:59.592679+0000","last_clean":"2024-08-16T19:08:59.592679+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:08:59.592679+0000","last_undegraded":"2024-08-16T19:08:59.592679+0000","last_fullsized":"2024-08-16T19:08:59.592679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457785+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:08:59.457785+0000","last_peered":"2024-08-16T19:08:59.457785+0000","last_clean":"2024-08-16T19:08:59.457785+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:08:59.457785+0000","last_undegraded":"2024-08-16T19:08:59.457785+0000","last_fullsized":"2024-08-16T19:08:59.457785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456321+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:08:59.456321+0000","last_peered":"2024-08-16T19:08:59.456321+0000","last_clean":"2024-08-16T19:08:59.456321+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:08:59.456321+0000","last_undegraded":"2024-08-16T19:08:59.456321+0000","last_fullsized":"2024-08-16T19:08:59.456321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454450+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:08:59.454450+0000","last_peered":"2024-08-16T19:08:59.454450+0000","last_clean":"2024-08-16T19:08:59.454450+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:08:59.454450+0000","last_undegraded":"2024-08-16T19:08:59.454450+0000","last_fullsized":"2024-08-16T19:08:59.454450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462131+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:08:59.462131+0000","last_peered":"2024-08-16T19:08:59.462131+0000","last_clean":"2024-08-16T19:08:59.462131+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:08:59.462131+0000","last_undegraded":"2024-08-16T19:08:59.462131+0000","last_fullsized":"2024-08-16T19:08:59.462131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593351+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:08:59.593351+0000","last_peered":"2024-08-16T19:08:59.593351+0000","last_clean":"2024-08-16T19:08:59.593351+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:08:59.593351+0000","last_undegraded":"2024-08-16T19:08:59.593351+0000","last_fullsized":"2024-08-16T19:08:59.593351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455179+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:08:59.455179+0000","last_peered":"2024-08-16T19:08:59.455179+0000","last_clean":"2024-08-16T19:08:59.455179+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:08:59.455179+0000","last_undegraded":"2024-08-16T19:08:59.455179+0000","last_fullsized":"2024-08-16T19:08:59.455179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456946+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:08:59.456946+0000","last_peered":"2024-08-16T19:08:59.456946+0000","last_clean":"2024-08-16T19:08:59.456946+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:08:59.456946+0000","last_undegraded":"2024-08-16T19:08:59.456946+0000","last_fullsized":"2024-08-16T19:08:59.456946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454562+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:08:59.454562+0000","last_peered":"2024-08-16T19:08:59.454562+0000","last_clean":"2024-08-16T19:08:59.454562+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:08:59.454562+0000","last_undegraded":"2024-08-16T19:08:59.454562+0000","last_fullsized":"2024-08-16T19:08:59.454562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457104+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:08:59.457104+0000","last_peered":"2024-08-16T19:08:59.457104+0000","last_clean":"2024-08-16T19:08:59.457104+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:08:59.457104+0000","last_undegraded":"2024-08-16T19:08:59.457104+0000","last_fullsized":"2024-08-16T19:08:59.457104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454682+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:08:59.454682+0000","last_peered":"2024-08-16T19:08:59.454682+0000","last_clean":"2024-08-16T19:08:59.454682+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:08:59.454682+0000","last_undegraded":"2024-08-16T19:08:59.454682+0000","last_fullsized":"2024-08-16T19:08:59.454682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458506+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:08:59.458506+0000","last_peered":"2024-08-16T19:08:59.458506+0000","last_clean":"2024-08-16T19:08:59.458506+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:08:59.458506+0000","last_undegraded":"2024-08-16T19:08:59.458506+0000","last_fullsized":"2024-08-16T19:08:59.458506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456800+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:08:59.456800+0000","last_peered":"2024-08-16T19:08:59.456800+0000","last_clean":"2024-08-16T19:08:59.456800+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:08:59.456800+0000","last_undegraded":"2024-08-16T19:08:59.456800+0000","last_fullsized":"2024-08-16T19:08:59.456800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462530+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:08:59.462530+0000","last_peered":"2024-08-16T19:08:59.462530+0000","last_clean":"2024-08-16T19:08:59.462530+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:08:59.462530+0000","last_undegraded":"2024-08-16T19:08:59.462530+0000","last_fullsized":"2024-08-16T19:08:59.462530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.461546+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:08:59.461546+0000","last_peered":"2024-08-16T19:08:59.461546+0000","last_clean":"2024-08-16T19:08:59.461546+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:08:59.461546+0000","last_undegraded":"2024-08-16T19:08:59.461546+0000","last_fullsized":"2024-08-16T19:08:59.461546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456328+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:08:59.456328+0000","last_peered":"2024-08-16T19:08:59.456328+0000","last_clean":"2024-08-16T19:08:59.456328+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:08:59.456328+0000","last_undegraded":"2024-08-16T19:08:59.456328+0000","last_fullsized":"2024-08-16T19:08:59.456328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457966+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:08:59.457966+0000","last_peered":"2024-08-16T19:08:59.457966+0000","last_clean":"2024-08-16T19:08:59.457966+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:08:59.457966+0000","last_undegraded":"2024-08-16T19:08:59.457966+0000","last_fullsized":"2024-08-16T19:08:59.457966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455554+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:08:59.455554+0000","last_peered":"2024-08-16T19:08:59.455554+0000","last_clean":"2024-08-16T19:08:59.455554+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:08:59.455554+0000","last_undegraded":"2024-08-16T19:08:59.455554+0000","last_fullsized":"2024-08-16T19:08:59.455554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462582+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:08:59.462582+0000","last_peered":"2024-08-16T19:08:59.462582+0000","last_clean":"2024-08-16T19:08:59.462582+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:08:59.462582+0000","last_undegraded":"2024-08-16T19:08:59.462582+0000","last_fullsized":"2024-08-16T19:08:59.462582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456804+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:08:59.456804+0000","last_peered":"2024-08-16T19:08:59.456804+0000","last_clean":"2024-08-16T19:08:59.456804+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:08:59.456804+0000","last_undegraded":"2024-08-16T19:08:59.456804+0000","last_fullsized":"2024-08-16T19:08:59.456804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593293+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:08:59.593293+0000","last_peered":"2024-08-16T19:08:59.593293+0000","last_clean":"2024-08-16T19:08:59.593293+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:08:59.593293+0000","last_undegraded":"2024-08-16T19:08:59.593293+0000","last_fullsized":"2024-08-16T19:08:59.593293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595085+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:08:59.595085+0000","last_peered":"2024-08-16T19:08:59.595085+0000","last_clean":"2024-08-16T19:08:59.595085+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:08:59.595085+0000","last_undegraded":"2024-08-16T19:08:59.595085+0000","last_fullsized":"2024-08-16T19:08:59.595085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593011+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:08:59.593011+0000","last_peered":"2024-08-16T19:08:59.593011+0000","last_clean":"2024-08-16T19:08:59.593011+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:08:59.593011+0000","last_undegraded":"2024-08-16T19:08:59.593011+0000","last_fullsized":"2024-08-16T19:08:59.593011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545609+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:08:59.545609+0000","last_peered":"2024-08-16T19:08:59.545609+0000","last_clean":"2024-08-16T19:08:59.545609+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:08:59.545609+0000","last_undegraded":"2024-08-16T19:08:59.545609+0000","last_fullsized":"2024-08-16T19:08:59.545609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455713+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:08:59.455713+0000","last_peered":"2024-08-16T19:08:59.455713+0000","last_clean":"2024-08-16T19:08:59.455713+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:08:59.455713+0000","last_undegraded":"2024-08-16T19:08:59.455713+0000","last_fullsized":"2024-08-16T19:08:59.455713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466874+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:08:59.466874+0000","last_peered":"2024-08-16T19:08:59.466874+0000","last_clean":"2024-08-16T19:08:59.466874+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:08:59.466874+0000","last_undegraded":"2024-08-16T19:08:59.466874+0000","last_fullsized":"2024-08-16T19:08:59.466874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456999+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:08:59.456999+0000","last_peered":"2024-08-16T19:08:59.456999+0000","last_clean":"2024-08-16T19:08:59.456999+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:08:59.456999+0000","last_undegraded":"2024-08-16T19:08:59.456999+0000","last_fullsized":"2024-08-16T19:08:59.456999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462773+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:08:59.462773+0000","last_peered":"2024-08-16T19:08:59.462773+0000","last_clean":"2024-08-16T19:08:59.462773+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:08:59.462773+0000","last_undegraded":"2024-08-16T19:08:59.462773+0000","last_fullsized":"2024-08-16T19:08:59.462773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545175+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:08:59.545175+0000","last_peered":"2024-08-16T19:08:59.545175+0000","last_clean":"2024-08-16T19:08:59.545175+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:08:59.545175+0000","last_undegraded":"2024-08-16T19:08:59.545175+0000","last_fullsized":"2024-08-16T19:08:59.545175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457436+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:08:59.457436+0000","last_peered":"2024-08-16T19:08:59.457436+0000","last_clean":"2024-08-16T19:08:59.457436+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:08:59.457436+0000","last_undegraded":"2024-08-16T19:08:59.457436+0000","last_fullsized":"2024-08-16T19:08:59.457436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457698+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:08:59.457698+0000","last_peered":"2024-08-16T19:08:59.457698+0000","last_clean":"2024-08-16T19:08:59.457698+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:08:59.457698+0000","last_undegraded":"2024-08-16T19:08:59.457698+0000","last_fullsized":"2024-08-16T19:08:59.457698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455841+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:08:59.455841+0000","last_peered":"2024-08-16T19:08:59.455841+0000","last_clean":"2024-08-16T19:08:59.455841+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:08:59.455841+0000","last_undegraded":"2024-08-16T19:08:59.455841+0000","last_fullsized":"2024-08-16T19:08:59.455841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457704+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:08:59.457704+0000","last_peered":"2024-08-16T19:08:59.457704+0000","last_clean":"2024-08-16T19:08:59.457704+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:08:59.457704+0000","last_undegraded":"2024-08-16T19:08:59.457704+0000","last_fullsized":"2024-08-16T19:08:59.457704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455744+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:08:59.455744+0000","last_peered":"2024-08-16T19:08:59.455744+0000","last_clean":"2024-08-16T19:08:59.455744+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:08:59.455744+0000","last_undegraded":"2024-08-16T19:08:59.455744+0000","last_fullsized":"2024-08-16T19:08:59.455744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457744+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:08:59.457744+0000","last_peered":"2024-08-16T19:08:59.457744+0000","last_clean":"2024-08-16T19:08:59.457744+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:08:59.457744+0000","last_undegraded":"2024-08-16T19:08:59.457744+0000","last_fullsized":"2024-08-16T19:08:59.457744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546015+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:08:59.546015+0000","last_peered":"2024-08-16T19:08:59.546015+0000","last_clean":"2024-08-16T19:08:59.546015+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:08:59.546015+0000","last_undegraded":"2024-08-16T19:08:59.546015+0000","last_fullsized":"2024-08-16T19:08:59.546015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593295+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:08:59.593295+0000","last_peered":"2024-08-16T19:08:59.593295+0000","last_clean":"2024-08-16T19:08:59.593295+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:08:59.593295+0000","last_undegraded":"2024-08-16T19:08:59.593295+0000","last_fullsized":"2024-08-16T19:08:59.593295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454659+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:08:59.454659+0000","last_peered":"2024-08-16T19:08:59.454659+0000","last_clean":"2024-08-16T19:08:59.454659+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:08:59.454659+0000","last_undegraded":"2024-08-16T19:08:59.454659+0000","last_fullsized":"2024-08-16T19:08:59.454659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456837+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:08:59.456837+0000","last_peered":"2024-08-16T19:08:59.456837+0000","last_clean":"2024-08-16T19:08:59.456837+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:08:59.456837+0000","last_undegraded":"2024-08-16T19:08:59.456837+0000","last_fullsized":"2024-08-16T19:08:59.456837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458460+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:08:59.458460+0000","last_peered":"2024-08-16T19:08:59.458460+0000","last_clean":"2024-08-16T19:08:59.458460+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:08:59.458460+0000","last_undegraded":"2024-08-16T19:08:59.458460+0000","last_fullsized":"2024-08-16T19:08:59.458460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457358+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:08:59.457358+0000","last_peered":"2024-08-16T19:08:59.457358+0000","last_clean":"2024-08-16T19:08:59.457358+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:08:59.457358+0000","last_undegraded":"2024-08-16T19:08:59.457358+0000","last_fullsized":"2024-08-16T19:08:59.457358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458036+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:08:59.458036+0000","last_peered":"2024-08-16T19:08:59.458036+0000","last_clean":"2024-08-16T19:08:59.458036+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:08:59.458036+0000","last_undegraded":"2024-08-16T19:08:59.458036+0000","last_fullsized":"2024-08-16T19:08:59.458036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457901+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:08:59.457901+0000","last_peered":"2024-08-16T19:08:59.457901+0000","last_clean":"2024-08-16T19:08:59.457901+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:08:59.457901+0000","last_undegraded":"2024-08-16T19:08:59.457901+0000","last_fullsized":"2024-08-16T19:08:59.457901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462449+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:08:59.462449+0000","last_peered":"2024-08-16T19:08:59.462449+0000","last_clean":"2024-08-16T19:08:59.462449+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:08:59.462449+0000","last_undegraded":"2024-08-16T19:08:59.462449+0000","last_fullsized":"2024-08-16T19:08:59.462449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64259,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":185,"num_read_kb":167,"num_write":242,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":192777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776156,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1152491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":512644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":508926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430250,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1105812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593775,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":504442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822707,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1100614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":508516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215164,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":503760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":562359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":562139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":505898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:09:04.736 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-16T19:09:04.736 INFO:tasks.cephadm:Setting up client nodes... 2024-08-16T19:09:04.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-16T19:09:05.001 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- 172.21.15.62:0/3829325479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0100840 msgr2=0x7f98d0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/3829325479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0100840 0x7f98d0100c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009900 tx=0x7f98cc02a410 comp rx=0 tx=0).stop 2024-08-16T19:09:05.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- 172.21.15.62:0/3829325479 shutdown_connections 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/3829325479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0102400 0x7f98d0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/3829325479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0101a40 0x7f98d0101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/3829325479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0100840 0x7f98d0100c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- 172.21.15.62:0/3829325479 >> 172.21.15.62:0/3829325479 conn(0x7f98d00fbfd0 msgr2=0x7f98d00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- 172.21.15.62:0/3829325479 shutdown_connections 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- 172.21.15.62:0/3829325479 wait complete. 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 Processor -- start 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.467+0000 7f98d7aa3640 1 -- start start 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0100840 0x7f98d010b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0101a40 0x7f98d0109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010d5f0 con 0x7f98d0102400 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010d470 con 0x7f98d0100840 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d7aa3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010d770 con 0x7f98d0101a40 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50466/0 (socket says 172.21.15.62:50466) 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 -- 172.21.15.62:0/605979251 learned_addr learned my addr 172.21.15.62:0/605979251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d583f640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0100840 0x7f98d010b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d503e640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0101a40 0x7f98d0109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 -- 172.21.15.62:0/605979251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0101a40 msgr2=0x7f98d0109810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0101a40 0x7f98d0109810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 -- 172.21.15.62:0/605979251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0100840 msgr2=0x7f98d010b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0100840 0x7f98d010b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 -- 172.21.15.62:0/605979251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d010a610 con 0x7f98d0102400 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.468+0000 7f98d6040640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f98c800cad0 tx=0x7f98c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.469+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800ae20 con 0x7f98d0102400 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.469+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c8005020 con 0x7f98d0102400 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.469+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d01a1df0 con 0x7f98d0102400 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.469+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c800f6f0 con 0x7f98d0102400 2024-08-16T19:09:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.469+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01a2250 con 0x7f98d0102400 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.470+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d0107c90 con 0x7f98d0102400 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.471+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f98c80051d0 con 0x7f98d0102400 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.471+0000 7f98c6ffd640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 0x7f98bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.472+0000 7f98d583f640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 0x7f98bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.472+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f98c80cf520 con 0x7f98d0102400 2024-08-16T19:09:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.472+0000 7f98d583f640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 0x7f98bc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005ab0 tx=0x7f98cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.476+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f98c8014030 con 0x7f98d0102400 2024-08-16T19:09:05.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.718+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f98d002d800 con 0x7f98d0102400 2024-08-16T19:09:05.719 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:05 smithi062 ceph-mon[32993]: pgmap v293: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-08-16T19:09:05.719 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:05 smithi062 ceph-mon[32993]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:05.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.721+0000 7f98c6ffd640 1 -- 172.21.15.62:0/605979251 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f98c809b970 con 0x7f98d0102400 2024-08-16T19:09:05.721 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-08-16T19:09:05.722 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDRo79m7SntKhAAth2RjVsBWhrWaYqPLNcMRA== 2024-08-16T19:09:05.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 msgr2=0x7f98bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:05.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 0x7f98bc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005ab0 tx=0x7f98cc000bc0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 msgr2=0x7f98d0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f98c800cad0 tx=0x7f98c800cf90 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 shutdown_connections 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98d0102400 0x7f98d0109d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98d0101a40 0x7f98d0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f98bc0a2c10 0x7f98bc0a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 --2- 172.21.15.62:0/605979251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98d0100840 0x7f98d010b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 >> 172.21.15.62:0/605979251 conn(0x7f98d00fbfd0 msgr2=0x7f98d0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 shutdown_connections 2024-08-16T19:09:05.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:05.724+0000 7f98d7aa3640 1 -- 172.21.15.62:0/605979251 wait complete. 2024-08-16T19:09:05.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:05 smithi155 ceph-mon[37413]: pgmap v293: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-08-16T19:09:05.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:05 smithi155 ceph-mon[37413]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:05 smithi161 ceph-mon[37576]: pgmap v293: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-08-16T19:09:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:05 smithi161 ceph-mon[37576]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:06.452 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:09:06.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-16T19:09:06.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-16T19:09:06.490 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-16T19:09:06.490 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-16T19:09:06.490 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-16T19:09:06.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:06 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-16T19:09:06.552 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:06 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-16T19:09:06.730 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:06 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-16T19:09:06.731 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:06 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-16T19:09:06.792 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:06 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-16T19:09:06.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:06 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/605979251' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-16T19:09:07.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 -- 172.21.15.62:0/1496598801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 msgr2=0x7f08e8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 --2- 172.21.15.62:0/1496598801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e8107c90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009930 tx=0x7f08d802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 -- 172.21.15.62:0/1496598801 shutdown_connections 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 --2- 172.21.15.62:0/1496598801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e8102250 0x7f08e810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 --2- 172.21.15.62:0/1496598801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e81018b0 0x7f08e8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 --2- 172.21.15.62:0/1496598801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e8107c90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 -- 172.21.15.62:0/1496598801 >> 172.21.15.62:0/1496598801 conn(0x7f08e80fd5f0 msgr2=0x7f08e80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 -- 172.21.15.62:0/1496598801 shutdown_connections 2024-08-16T19:09:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.249+0000 7f08ed25f640 1 -- 172.21.15.62:0/1496598801 wait complete. 2024-08-16T19:09:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.250+0000 7f08ed25f640 1 Processor -- start 2024-08-16T19:09:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.250+0000 7f08ed25f640 1 -- start start 2024-08-16T19:09:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.250+0000 7f08ed25f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e81018b0 0x7f08e8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.250+0000 7f08ed25f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 0x7f08e8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.250+0000 7f08ed25f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08ed25f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111da0 con 0x7f08e81078b0 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08ed25f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111c20 con 0x7f08e8102250 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08ed25f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08e8111f20 con 0x7f08e81018b0 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50492/0 (socket says 172.21.15.62:50492) 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 0x7f08e8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 -- 172.21.15.62:0/2996732740 learned_addr learned my addr 172.21.15.62:0/2996732740 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e6d9d640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e81018b0 0x7f08e8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 -- 172.21.15.62:0/2996732740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e81018b0 msgr2=0x7f08e8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e81018b0 0x7f08e8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 -- 172.21.15.62:0/2996732740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 msgr2=0x7f08e8198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 0x7f08e8198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 -- 172.21.15.62:0/2996732740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e819d2d0 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e659c640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 0x7f08e8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08e759e640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b970 tx=0x7f08e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.251+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0019070 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.252+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e0014440 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.252+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e819d5c0 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.252+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00075d0 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.252+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e81a4ea0 con 0x7f08e81078b0 2024-08-16T19:09:07.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.253+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e8101d90 con 0x7f08e81078b0 2024-08-16T19:09:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.254+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f08e001c070 con 0x7f08e81078b0 2024-08-16T19:09:07.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.254+0000 7f08cffff640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 0x7f08d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:07.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.255+0000 7f08e6d9d640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 0x7f08d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:07.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.255+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f08e00ce9c0 con 0x7f08e81078b0 2024-08-16T19:09:07.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.255+0000 7f08e6d9d640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 0x7f08d00a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006010 tx=0x7f08d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:07.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.259+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f08e009af20 con 0x7f08e81078b0 2024-08-16T19:09:07.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.486+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f08e8109e80 con 0x7f08e81078b0 2024-08-16T19:09:07.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.491+0000 7f08cffff640 1 -- 172.21.15.62:0/2996732740 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263194 (secure 0 0 0) 0x7f08e009f220 con 0x7f08e81078b0 2024-08-16T19:09:07.494 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:07.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.494+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 msgr2=0x7f08d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.494+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 0x7f08d00a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006010 tx=0x7f08d8000bc0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.494+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 msgr2=0x7f08e819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.494+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b970 tx=0x7f08e000bd70 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 shutdown_connections 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08e81078b0 0x7f08e819cbf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f08d00a2c10 0x7f08d00a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e8102250 0x7f08e8198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 --2- 172.21.15.62:0/2996732740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e81018b0 0x7f08e8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 >> 172.21.15.62:0/2996732740 conn(0x7f08e80fd5f0 msgr2=0x7f08e80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 shutdown_connections 2024-08-16T19:09:07.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:07.495+0000 7f08ed25f640 1 -- 172.21.15.62:0/2996732740 wait complete. 2024-08-16T19:09:07.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:07 smithi062 ceph-mon[32993]: pgmap v294: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-16T19:09:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:07 smithi155 ceph-mon[37413]: pgmap v294: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-16T19:09:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:07 smithi161 ceph-mon[37576]: pgmap v294: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-08-16T19:09:08.251 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":727495888},{"type":"v1","addr":"172.21.15.62:6801","nonce":727495888}]},"active_addr":"172.21.15.62:6801/727495888","active_change":"2024-08-16T19:01:57.573631+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.62:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":795341867}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":439686452}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":1715350407}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":151448061}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.62:0","nonce":890362399}]}]} 2024-08-16T19:09:08.257 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-16T19:09:08.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-16T19:09:08.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:09:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:08 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2996732740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-16T19:09:08.520 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:08.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:08 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2996732740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-16T19:09:08.889 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:08 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2996732740' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-16T19:09:08.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 -- 172.21.15.62:0/1940192647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081007b0 msgr2=0x7f0308100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:08.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 --2- 172.21.15.62:0/1940192647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081007b0 0x7f0308100c10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009930 tx=0x7f02f002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:08.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 -- 172.21.15.62:0/1940192647 shutdown_connections 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 --2- 172.21.15.62:0/1940192647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0308101150 0x7f030810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 --2- 172.21.15.62:0/1940192647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081007b0 0x7f0308100c10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 --2- 172.21.15.62:0/1940192647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03081067b0 0x7f0308106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 -- 172.21.15.62:0/1940192647 >> 172.21.15.62:0/1940192647 conn(0x7f03080fc4d0 msgr2=0x7f03080fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 -- 172.21.15.62:0/1940192647 shutdown_connections 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.958+0000 7f030d19a640 1 -- 172.21.15.62:0/1940192647 wait complete. 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.959+0000 7f030d19a640 1 Processor -- start 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.959+0000 7f030d19a640 1 -- start start 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 0x7f03081a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0308101150 0x7f03081a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0308110bb0 con 0x7f03081067b0 2024-08-16T19:09:08.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0308110a30 con 0x7f03081007b0 2024-08-16T19:09:08.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030d19a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0308110d30 con 0x7f0308101150 2024-08-16T19:09:08.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:08.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f0306d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 0x7f03081a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50506/0 (socket says 172.21.15.62:50506) 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f0306d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 0x7f03081a3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:40098/0 (socket says 172.21.15.62:40098) 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 -- 172.21.15.62:0/3962560083 learned_addr learned my addr 172.21.15.62:0/3962560083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 -- 172.21.15.62:0/3962560083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0308101150 msgr2=0x7f03081a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0308101150 0x7f03081a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 -- 172.21.15.62:0/3962560083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 msgr2=0x7f03081a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 0x7f03081a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.960+0000 7f030759e640 1 -- 172.21.15.62:0/3962560083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030819efb0 con 0x7f03081067b0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f030759e640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f030000cce0 tx=0x7f030000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000f450 con 0x7f03081067b0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030000fad0 con 0x7f03081067b0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030819f2a0 con 0x7f03081067b0 2024-08-16T19:09:08.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300018c60 con 0x7f03081067b0 2024-08-16T19:09:08.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.961+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03081ab240 con 0x7f03081067b0 2024-08-16T19:09:08.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.962+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0308100c10 con 0x7f03081067b0 2024-08-16T19:09:08.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.964+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f030000fc80 con 0x7f03081067b0 2024-08-16T19:09:08.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.964+0000 7f0304d99640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 0x7f02f40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:08.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.964+0000 7f0306d9d640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 0x7f02f40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:08.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.964+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f03000cfa00 con 0x7f03081067b0 2024-08-16T19:09:08.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.965+0000 7f0306d9d640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 0x7f02f40a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02f8005fd0 tx=0x7f02f800bf30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:08.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:08.968+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03000d1e20 con 0x7f03081067b0 2024-08-16T19:09:09.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.146+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03081ab5d0 con 0x7f03081067b0 2024-08-16T19:09:09.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.148+0000 7f0304d99640 1 -- 172.21.15.62:0/3962560083 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24632 (secure 0 0 0) 0x7f0300014030 con 0x7f03081067b0 2024-08-16T19:09:09.149 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:09.149 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:59.449917+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:09:09.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 msgr2=0x7f02f40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 0x7f02f40a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02f8005fd0 tx=0x7f02f800bf30 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 msgr2=0x7f030819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f030000cce0 tx=0x7f030000a620 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 shutdown_connections 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03081067b0 0x7f030819e830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0308101150 0x7f03081a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f02f40a2c90 0x7f02f40a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 --2- 172.21.15.62:0/3962560083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03081007b0 0x7f03081a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 >> 172.21.15.62:0/3962560083 conn(0x7f03080fc4d0 msgr2=0x7f030806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 shutdown_connections 2024-08-16T19:09:09.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:09.150+0000 7f030d19a640 1 -- 172.21.15.62:0/3962560083 wait complete. 2024-08-16T19:09:09.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:09 smithi062 ceph-mon[32993]: pgmap v295: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-08-16T19:09:09.523 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:09 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3962560083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:09 smithi155 ceph-mon[37413]: pgmap v295: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-08-16T19:09:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:09 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3962560083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:09 smithi161 ceph-mon[37576]: pgmap v295: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-08-16T19:09:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:09 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3962560083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:10.008 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-16T19:09:10.008 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:09:10.273 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- 172.21.15.62:0/1599872277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01078b0 msgr2=0x7f69d0107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/1599872277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01078b0 0x7f69d0107c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009930 tx=0x7f69b802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- 172.21.15.62:0/1599872277 shutdown_connections 2024-08-16T19:09:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/1599872277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d0102250 0x7f69d010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/1599872277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d01018b0 0x7f69d0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/1599872277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01078b0 0x7f69d0107c90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- 172.21.15.62:0/1599872277 >> 172.21.15.62:0/1599872277 conn(0x7f69d00fd5f0 msgr2=0x7f69d00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- 172.21.15.62:0/1599872277 shutdown_connections 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- 172.21.15.62:0/1599872277 wait complete. 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 Processor -- start 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.736+0000 7f69d6cc2640 1 -- start start 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d0102250 0x7f69d0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 0x7f69d019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111d50 con 0x7f69d01018b0 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111bd0 con 0x7f69d0102250 2024-08-16T19:09:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d6cc2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0111ed0 con 0x7f69d01078b0 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50528/0 (socket says 172.21.15.62:50528) 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 -- 172.21.15.62:0/2194358719 learned_addr learned my addr 172.21.15.62:0/2194358719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69cffff640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d0102250 0x7f69d0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d525f640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 0x7f69d019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 -- 172.21.15.62:0/2194358719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 msgr2=0x7f69d019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 0x7f69d019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 -- 172.21.15.62:0/2194358719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d0102250 msgr2=0x7f69d0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d0102250 0x7f69d0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 -- 172.21.15.62:0/2194358719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d019d2c0 con 0x7f69d01018b0 2024-08-16T19:09:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d525f640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 0x7f69d019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.737+0000 7f69d4a5e640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009a50 tx=0x7f69b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.738+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b803f030 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.738+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b8003ce0 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.738+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d019d550 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.738+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8035070 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.738+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01a4e90 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.739+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d0101d90 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.741+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69b8042030 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.742+0000 7f69cdffb640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 0x7f69bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.742+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f69b80efe30 con 0x7f69d01018b0 2024-08-16T19:09:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.742+0000 7f69cffff640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 0x7f69bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:10.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.742+0000 7f69cffff640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 0x7f69bc0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69d00ffc50 tx=0x7f69c000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:10.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.745+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f69b80bc8d0 con 0x7f69d01018b0 2024-08-16T19:09:10.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.924+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69d0109e20 con 0x7f69d01018b0 2024-08-16T19:09:10.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.926+0000 7f69cdffb640 1 -- 172.21.15.62:0/2194358719 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24632 (secure 0 0 0) 0x7f69b80bc1d0 con 0x7f69d01018b0 2024-08-16T19:09:10.927 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:10.927 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:59.449917+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 msgr2=0x7f69bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 0x7f69bc0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f69d00ffc50 tx=0x7f69c000d040 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 msgr2=0x7f69d0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009a50 tx=0x7f69b8004270 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 shutdown_connections 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69d01078b0 0x7f69d019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f69bc0a2c90 0x7f69bc0a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69d0102250 0x7f69d0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 --2- 172.21.15.62:0/2194358719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69d01018b0 0x7f69d0198310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:10.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 >> 172.21.15.62:0/2194358719 conn(0x7f69d00fd5f0 msgr2=0x7f69d00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:10.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 shutdown_connections 2024-08-16T19:09:10.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:10.929+0000 7f69d6cc2640 1 -- 172.21.15.62:0/2194358719 wait complete. 2024-08-16T19:09:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:11 smithi062 ceph-mon[32993]: pgmap v296: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-16T19:09:11.727 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:11 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2194358719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:11.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-16T19:09:11.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-16T19:09:11.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-16T19:09:11.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-16T19:09:11.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-16T19:09:11.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-16T19:09:11.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-16T19:09:11.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-16T19:09:11.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-16T19:09:11.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-16T19:09:11.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-16T19:09:11.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-16T19:09:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:11 smithi155 ceph-mon[37413]: pgmap v296: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-16T19:09:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:11 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2194358719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:11 smithi161 ceph-mon[37576]: pgmap v296: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-08-16T19:09:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:11 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2194358719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:12.460 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:12.466 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:12.521 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 -- 172.21.15.62:0/820953810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 msgr2=0x7ffa9c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/820953810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c10e780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009cd0 tx=0x7ffa9402a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 -- 172.21.15.62:0/820953810 shutdown_connections 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/820953810 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c10e780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/820953810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9c1018b0 0x7ffa9c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/820953810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1078b0 0x7ffa9c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 -- 172.21.15.62:0/820953810 >> 172.21.15.62:0/820953810 conn(0x7ffa9c0fd5f0 msgr2=0x7ffa9c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 -- 172.21.15.62:0/820953810 shutdown_connections 2024-08-16T19:09:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.159+0000 7ffaa22fd640 1 -- 172.21.15.62:0/820953810 wait complete. 2024-08-16T19:09:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 Processor -- start 2024-08-16T19:09:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 -- start start 2024-08-16T19:09:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9c1018b0 0x7ffa9c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1078b0 0x7ffa9c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111d50 con 0x7ffa9c102250 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111bd0 con 0x7ffa9c1018b0 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffaa22fd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa9c111ed0 con 0x7ffa9c1078b0 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffa9b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34300/0 (socket says 172.21.15.62:34300) 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffa9b7fe640 1 -- 172.21.15.62:0/1242658944 learned_addr learned my addr 172.21.15.62:0/1242658944 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.160+0000 7ffa9bfff640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9c1018b0 0x7ffa9c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffaa089a640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1078b0 0x7ffa9c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 -- 172.21.15.62:0/1242658944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1078b0 msgr2=0x7ffa9c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c1078b0 0x7ffa9c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 -- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9c1018b0 msgr2=0x7ffa9c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa9c1018b0 0x7ffa9c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9c19d280 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa9b7fe640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c198810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b4a0 tx=0x7ffa9000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90015020 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9c19d570 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa90011d10 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000f2e0 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.161+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa9c1a4e50 con 0x7ffa9c102250 2024-08-16T19:09:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.162+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa9c0626b0 con 0x7ffa9c102250 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.163+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa90018020 con 0x7ffa9c102250 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa997fa640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa840a2c10 0x7ffa840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7ffa900cf2b0 con 0x7ffa9c102250 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa9bfff640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa840a2c10 0x7ffa840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa997fa640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] conn(0x7ffa840adde0 0x7ffa840b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa90004450 con 0x7ffa840adde0 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.164+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ffa900cf700 con 0x7ffa9c102250 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.165+0000 7ffa9bfff640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa840a2c10 0x7ffa840a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009990 tx=0x7ffa8c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.165+0000 7ffaa089a640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] conn(0x7ffa840adde0 0x7ffa840b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.165+0000 7ffaa089a640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] conn(0x7ffa840adde0 0x7ffa840b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.166+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== osd.7 v2:172.21.15.155:6816/785303840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa90004450 con 0x7ffa840adde0 2024-08-16T19:09:13.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.180+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 --> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa9c062ab0 con 0x7ffa840adde0 2024-08-16T19:09:13.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.180+0000 7ffa997fa640 1 -- 172.21.15.62:0/1242658944 <== osd.7 v2:172.21.15.155:6816/785303840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa9c062ab0 con 0x7ffa840adde0 2024-08-16T19:09:13.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] conn(0x7ffa840adde0 msgr2=0x7ffa840b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] conn(0x7ffa840adde0 0x7ffa840b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa840a2c10 msgr2=0x7ffa840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7ffa840a2c10 0x7ffa840a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009990 tx=0x7ffa8c009110 comp rx=0 tx=0).stop 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 msgr2=0x7ffa9c198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 --2- 172.21.15.62:0/1242658944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa9c102250 0x7ffa9c198810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000b4a0 tx=0x7ffa9000b8a0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa089a640 1 -- 172.21.15.62:0/1242658944 reap_dead start 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 shutdown_connections 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 >> 172.21.15.62:0/1242658944 conn(0x7ffa9c0fd5f0 msgr2=0x7ffa9c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:13.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 shutdown_connections 2024-08-16T19:09:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.181+0000 7ffaa22fd640 1 -- 172.21.15.62:0/1242658944 wait complete. 2024-08-16T19:09:13.717 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:13 smithi062 ceph-mon[32993]: pgmap v297: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 -- 172.21.15.62:0/1963767137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80742c0 msgr2=0x7f7cf80746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 --2- 172.21.15.62:0/1963767137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80742c0 0x7f7cf80746a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009900 tx=0x7f7cf402a480 comp rx=0 tx=0).stop 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 -- 172.21.15.62:0/1963767137 shutdown_connections 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 --2- 172.21.15.62:0/1963767137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf81128c0 0x7f7cf8114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 --2- 172.21.15.62:0/1963767137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf8074be0 0x7f7cf8112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.798+0000 7f7cff778640 1 --2- 172.21.15.62:0/1963767137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf80742c0 0x7f7cf80746a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.799+0000 7f7cff778640 1 -- 172.21.15.62:0/1963767137 >> 172.21.15.62:0/1963767137 conn(0x7f7cf80fd5f0 msgr2=0x7f7cf80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.799+0000 7f7cff778640 1 -- 172.21.15.62:0/1963767137 shutdown_connections 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.799+0000 7f7cff778640 1 -- 172.21.15.62:0/1963767137 wait complete. 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.799+0000 7f7cff778640 1 Processor -- start 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.799+0000 7f7cff778640 1 -- start start 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf80742c0 0x7f7cf81a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 0x7f7cf81a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf81128c0 0x7f7cf819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81182a0 con 0x7f7cf8074be0 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8118120 con 0x7f7cf80742c0 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8118420 con 0x7f7cf81128c0 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 0x7f7cf81a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 0x7f7cf81a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34310/0 (socket says 172.21.15.62:34310) 2024-08-16T19:09:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 -- 172.21.15.62:0/1530833150 learned_addr learned my addr 172.21.15.62:0/1530833150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfdd15640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf81128c0 0x7f7cf819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfd514640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf80742c0 0x7f7cf81a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 -- 172.21.15.62:0/1530833150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf81128c0 msgr2=0x7f7cf819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cf81128c0 0x7f7cf819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 -- 172.21.15.62:0/1530833150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf80742c0 msgr2=0x7f7cf81a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cf80742c0 0x7f7cf81a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf819ed20 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cfcd13640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 0x7f7cf81a4970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800d950 tx=0x7f7ce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8017030 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce8004020 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf819f010 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80047a0 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.800+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81ab300 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.801+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cf80ff810 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.803+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7ce80041d0 con 0x7f7cf8074be0 2024-08-16T19:09:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.803+0000 7f7cee7fc640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7ce40a2c10 0x7f7ce40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cfd514640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7ce40a2c10 0x7f7ce40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7ce80cf690 con 0x7f7cf8074be0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cee7fc640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] conn(0x7f7ce40adde0 0x7f7ce40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ce40b0950 con 0x7f7ce40adde0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7ce80cfae0 con 0x7f7cf8074be0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cfdd15640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] conn(0x7f7ce40adde0 0x7f7ce40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cfd514640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7ce40a2c10 0x7f7ce40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005ab0 tx=0x7f7cf4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.804+0000 7f7cfdd15640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] conn(0x7f7ce40adde0 0x7f7ce40b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.805+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== osd.3 v2:172.21.15.62:6810/1745722345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ce40b0950 con 0x7f7ce40adde0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.818+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 --> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7cf8075a00 con 0x7f7ce40adde0 2024-08-16T19:09:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cee7fc640 1 -- 172.21.15.62:0/1530833150 <== osd.3 v2:172.21.15.62:6810/1745722345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7cf8075a00 con 0x7f7ce40adde0 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] conn(0x7f7ce40adde0 msgr2=0x7f7ce40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] conn(0x7f7ce40adde0 0x7f7ce40b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7ce40a2c10 msgr2=0x7f7ce40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7ce40a2c10 0x7f7ce40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005ab0 tx=0x7f7cf4005a20 comp rx=0 tx=0).stop 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 msgr2=0x7f7cf81a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 --2- 172.21.15.62:0/1530833150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf8074be0 0x7f7cf81a4970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800d950 tx=0x7f7ce800de10 comp rx=0 tx=0).stop 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cfdd15640 1 -- 172.21.15.62:0/1530833150 reap_dead start 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 shutdown_connections 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 >> 172.21.15.62:0/1530833150 conn(0x7f7cf80fd5f0 msgr2=0x7f7cf80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 shutdown_connections 2024-08-16T19:09:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:13.819+0000 7f7cff778640 1 -- 172.21.15.62:0/1530833150 wait complete. 2024-08-16T19:09:13.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:13 smithi155 ceph-mon[37413]: pgmap v297: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-16T19:09:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:13 smithi161 ceph-mon[37576]: pgmap v297: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-08-16T19:09:14.493 INFO:teuthology.orchestra.run.smithi062.stdout:103079215167 2024-08-16T19:09:14.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-16T19:09:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 -- 172.21.15.62:0/2409315654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 msgr2=0x7fae48100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/2409315654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae48100f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009900 tx=0x7fae3c02a480 comp rx=0 tx=0).stop 2024-08-16T19:09:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 -- 172.21.15.62:0/2409315654 shutdown_connections 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/2409315654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 0x7fae4810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/2409315654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae481022a0 0x7fae481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/2409315654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae48100f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 -- 172.21.15.62:0/2409315654 >> 172.21.15.62:0/2409315654 conn(0x7fae480fc500 msgr2=0x7fae480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 -- 172.21.15.62:0/2409315654 shutdown_connections 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.657+0000 7fae4fd9f640 1 -- 172.21.15.62:0/2409315654 wait complete. 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 Processor -- start 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 -- start start 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae481971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae481022a0 0x7fae481976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 0x7fae4819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:14.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110c70 con 0x7fae48100b50 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110af0 con 0x7fae481022a0 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4fd9f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110df0 con 0x7fae48102c20 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4db3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae481971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4db3b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae481971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34332/0 (socket says 172.21.15.62:34332) 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4db3b640 1 -- 172.21.15.62:0/3122771892 learned_addr learned my addr 172.21.15.62:0/3122771892 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.658+0000 7fae4d33a640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae481022a0 0x7fae481976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 -- 172.21.15.62:0/3122771892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 msgr2=0x7fae4819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4e33c640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 0x7fae4819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 0x7fae4819ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 -- 172.21.15.62:0/3122771892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae481022a0 msgr2=0x7fae481976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae481022a0 0x7fae481976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4819c160 con 0x7fae48100b50 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4e33c640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae48102c20 0x7fae4819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:14.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4db3b640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae481971b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c03f070 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae3c031c30 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4819c3f0 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c035030 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.659+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae481a3d30 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.660+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae480626b0 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.662+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae3c042070 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae3affd640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fae340a2c90 0x7fae340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae4d33a640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fae340a2c90 0x7fae340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fae3c0f05e0 con 0x7fae48100b50 2024-08-16T19:09:14.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae3affd640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] conn(0x7fae340ade60 0x7fae340b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae340b09d0 con 0x7fae340ade60 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fae3c0f0a20 con 0x7fae48100b50 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae4e33c640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] conn(0x7fae340ade60 0x7fae340b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.663+0000 7fae4d33a640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fae340a2c90 0x7fae340a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae44005fd0 tx=0x7fae44005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.664+0000 7fae4e33c640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] conn(0x7fae340ade60 0x7fae340b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:14.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.664+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== osd.9 v2:172.21.15.62:6826/851969557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae340b09d0 con 0x7fae340ade60 2024-08-16T19:09:14.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.678+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 --> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae48062ab0 con 0x7fae340ade60 2024-08-16T19:09:14.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae3affd640 1 -- 172.21.15.62:0/3122771892 <== osd.9 v2:172.21.15.62:6826/851969557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fae48062ab0 con 0x7fae340ade60 2024-08-16T19:09:14.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] conn(0x7fae340ade60 msgr2=0x7fae340b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] conn(0x7fae340ade60 0x7fae340b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fae340a2c90 msgr2=0x7fae340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fae340a2c90 0x7fae340a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae44005fd0 tx=0x7fae44005850 comp rx=0 tx=0).stop 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 msgr2=0x7fae481971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 --2- 172.21.15.62:0/3122771892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae48100b50 0x7fae481971b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c003c10 comp rx=0 tx=0).stop 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4e33c640 1 -- 172.21.15.62:0/3122771892 reap_dead start 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 shutdown_connections 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 >> 172.21.15.62:0/3122771892 conn(0x7fae480fc500 msgr2=0x7fae480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 shutdown_connections 2024-08-16T19:09:14.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:14.679+0000 7fae4fd9f640 1 -- 172.21.15.62:0/3122771892 wait complete. 2024-08-16T19:09:15.427 INFO:teuthology.orchestra.run.smithi062.stdout:270582939684 2024-08-16T19:09:15.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-16T19:09:15.496 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:15.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:15 smithi062 ceph-mon[32993]: pgmap v298: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-08-16T19:09:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:15 smithi155 ceph-mon[37413]: pgmap v298: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-08-16T19:09:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:15 smithi161 ceph-mon[37576]: pgmap v298: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-08-16T19:09:15.987 INFO:teuthology.orchestra.run.smithi062.stdout:206158430253 2024-08-16T19:09:15.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-16T19:09:16.029 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.029 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.034 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.034 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.036 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.039 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:16.043 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:17.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- 172.21.15.62:0/2403678073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344103390 msgr2=0x7f434410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 --2- 172.21.15.62:0/2403678073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344103390 0x7f434410b880 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4340009900 tx=0x7f434002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- 172.21.15.62:0/2403678073 shutdown_connections 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 --2- 172.21.15.62:0/2403678073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f434410bea0 0x7f434410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 --2- 172.21.15.62:0/2403678073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344103390 0x7f434410b880 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 --2- 172.21.15.62:0/2403678073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4344102a70 0x7f4344102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- 172.21.15.62:0/2403678073 >> 172.21.15.62:0/2403678073 conn(0x7f43440fc4d0 msgr2=0x7f43440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- 172.21.15.62:0/2403678073 shutdown_connections 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- 172.21.15.62:0/2403678073 wait complete. 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 Processor -- start 2024-08-16T19:09:17.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.263+0000 7f434b99f640 1 -- start start 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 0x7f43441981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4344103390 0x7f43441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f434410bea0 0x7f434419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111ca0 con 0x7f4344102a70 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111b20 con 0x7f434410bea0 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434b99f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4344111e20 con 0x7f4344103390 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 0x7f43441981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 0x7f43441981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34346/0 (socket says 172.21.15.62:34346) 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 -- 172.21.15.62:0/1053647940 learned_addr learned my addr 172.21.15.62:0/1053647940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f4349f3c640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f434410bea0 0x7f434419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f4348f3a640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4344103390 0x7f43441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 -- 172.21.15.62:0/1053647940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4344103390 msgr2=0x7f43441986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4344103390 0x7f43441986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 -- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f434410bea0 msgr2=0x7f434419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f434410bea0 0x7f434419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419d150 con 0x7f4344102a70 2024-08-16T19:09:17.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.264+0000 7f434973b640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 0x7f43441981a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f433800d950 tx=0x7f433800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338017070 con 0x7f4344102a70 2024-08-16T19:09:17.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4338009c30 con 0x7f4344102a70 2024-08-16T19:09:17.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434419d440 con 0x7f4344102a70 2024-08-16T19:09:17.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433800f6b0 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a4d20 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.265+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43440626b0 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.267+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f43380109d0 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.267+0000 7f43367fc640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f43300a2c90 0x7f43300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f4348f3a640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f43300a2c90 0x7f43300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f43380cf450 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f43367fc640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] conn(0x7f43300ade60 0x7f43300b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f433801de50 con 0x7f43300ade60 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f43380cf8a0 con 0x7f4344102a70 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f4349f3c640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] conn(0x7f43300ade60 0x7f43300b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f4348f3a640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f43300a2c90 0x7f43300a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4340009230 tx=0x7f43400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.268+0000 7f4349f3c640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] conn(0x7f43300ade60 0x7f43300b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.269+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== osd.4 v2:172.21.15.155:6808/34440659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f433801de50 con 0x7f43300ade60 2024-08-16T19:09:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.283+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 --> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4344062b30 con 0x7f43300ade60 2024-08-16T19:09:17.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f43367fc640 1 -- 172.21.15.62:0/1053647940 <== osd.4 v2:172.21.15.155:6808/34440659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4344062b30 con 0x7f43300ade60 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] conn(0x7f43300ade60 msgr2=0x7f43300b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] conn(0x7f43300ade60 0x7f43300b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f43300a2c90 msgr2=0x7f43300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f43300a2c90 0x7f43300a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4340009230 tx=0x7f43400058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 msgr2=0x7f43441981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 --2- 172.21.15.62:0/1053647940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344102a70 0x7f43441981a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f433800d950 tx=0x7f433800de10 comp rx=0 tx=0).stop 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f4349f3c640 1 -- 172.21.15.62:0/1053647940 reap_dead start 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 shutdown_connections 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 >> 172.21.15.62:0/1053647940 conn(0x7f43440fc4d0 msgr2=0x7f43440fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 shutdown_connections 2024-08-16T19:09:17.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.284+0000 7f434b99f640 1 -- 172.21.15.62:0/1053647940 wait complete. 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 -- 172.21.15.62:0/3049617326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 msgr2=0x7f67b0109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 --2- 172.21.15.62:0/3049617326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b0109380 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009900 tx=0x7f67a402a410 comp rx=0 tx=0).stop 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 -- 172.21.15.62:0/3049617326 shutdown_connections 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 --2- 172.21.15.62:0/3049617326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67b01098c0 0x7f67b01155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 --2- 172.21.15.62:0/3049617326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b0109380 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 --2- 172.21.15.62:0/3049617326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b010e8d0 0x7f67b010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 -- 172.21.15.62:0/3049617326 >> 172.21.15.62:0/3049617326 conn(0x7f67b006e000 msgr2=0x7f67b006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 -- 172.21.15.62:0/3049617326 shutdown_connections 2024-08-16T19:09:17.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.442+0000 7f67b52f8640 1 -- 172.21.15.62:0/3049617326 wait complete. 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.443+0000 7f67b52f8640 1 Processor -- start 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.443+0000 7f67b52f8640 1 -- start start 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b01a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b01098c0 0x7f67b01a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67b010e8d0 0x7f67b019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0118ad0 con 0x7f67b0108f20 2024-08-16T19:09:17.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0118950 con 0x7f67b01098c0 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67b52f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0118c50 con 0x7f67b010e8d0 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b01a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b01a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34370/0 (socket says 172.21.15.62:34370) 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 -- 172.21.15.62:0/1824729427 learned_addr learned my addr 172.21.15.62:0/1824729427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67ae7fc640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b01098c0 0x7f67b01a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67af7fe640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67b010e8d0 0x7f67b019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 -- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67b010e8d0 msgr2=0x7f67b019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67b010e8d0 0x7f67b019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 -- 172.21.15.62:0/1824729427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b01098c0 msgr2=0x7f67b01a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67b01098c0 0x7f67b01a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.444+0000 7f67aeffd640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01a0550 con 0x7f67b0108f20 2024-08-16T19:09:17.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f67aeffd640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b01a54c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c890 tx=0x7f67a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004220 con 0x7f67b0108f20 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b01a0840 con 0x7f67b0108f20 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a00043c0 con 0x7f67b0108f20 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004a00 con 0x7f67b0108f20 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.445+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b019ecf0 con 0x7f67b0108f20 2024-08-16T19:09:17.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.446+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67b019f330 con 0x7f67b0108f20 2024-08-16T19:09:17.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.448+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67a00079b0 con 0x7f67b0108f20 2024-08-16T19:09:17.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.448+0000 7f6797fff640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f67980a4ea0 0x7f67980a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.448+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f67a00cf150 con 0x7f67b0108f20 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.448+0000 7f67ae7fc640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f67980a4ea0 0x7f67980a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.449+0000 7f6797fff640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] conn(0x7f67980b0070 0x7f67980b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.449+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67980b2be0 con 0x7f67980b0070 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.449+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f67a00d6050 con 0x7f67b0108f20 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.449+0000 7f67ae7fc640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f67980a4ea0 0x7f67980a7360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.449+0000 7f67af7fe640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] conn(0x7f67980b0070 0x7f67980b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:17.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.450+0000 7f67af7fe640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] conn(0x7f67980b0070 0x7f67980b2510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:17.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.450+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== osd.2 v2:172.21.15.161:6800/3525036557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67980b2be0 con 0x7f67980b0070 2024-08-16T19:09:17.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.465+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 --> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67b019f730 con 0x7f67980b0070 2024-08-16T19:09:17.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f6797fff640 1 -- 172.21.15.62:0/1824729427 <== osd.2 v2:172.21.15.161:6800/3525036557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f67b019f730 con 0x7f67980b0070 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] conn(0x7f67980b0070 msgr2=0x7f67980b2510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] conn(0x7f67980b0070 0x7f67980b2510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f67980a4ea0 msgr2=0x7f67980a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f67980a4ea0 0x7f67980a7360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67a4000c00 tx=0x7f67a40058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 msgr2=0x7f67b01a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 --2- 172.21.15.62:0/1824729427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67b0108f20 0x7f67b01a54c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c890 tx=0x7f67a000cd50 comp rx=0 tx=0).stop 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67af7fe640 1 -- 172.21.15.62:0/1824729427 reap_dead start 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 shutdown_connections 2024-08-16T19:09:17.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.466+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 >> 172.21.15.62:0/1824729427 conn(0x7f67b006e000 msgr2=0x7f67b0072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:17.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.467+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 shutdown_connections 2024-08-16T19:09:17.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:17.467+0000 7f67b52f8640 1 -- 172.21.15.62:0/1824729427 wait complete. 2024-08-16T19:09:17.710 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:17 smithi062 ceph-mon[32993]: pgmap v299: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-16T19:09:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:17 smithi155 ceph-mon[37413]: pgmap v299: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-16T19:09:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:17 smithi161 ceph-mon[37576]: pgmap v299: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 -- 172.21.15.62:0/1474538055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 msgr2=0x7fe3d01078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/1474538055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d01078c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009930 tx=0x7fe3b802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 -- 172.21.15.62:0/1474538055 shutdown_connections 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/1474538055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0107fd0 0x7fe3d010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/1474538055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d01078c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/1474538055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0102a60 0x7fe3d0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 -- 172.21.15.62:0/1474538055 >> 172.21.15.62:0/1474538055 conn(0x7fe3d00fc500 msgr2=0x7fe3d00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.114+0000 7fe3d5b74640 1 -- 172.21.15.62:0/1474538055 shutdown_connections 2024-08-16T19:09:18.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.115+0000 7fe3d5b74640 1 -- 172.21.15.62:0/1474538055 wait complete. 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.115+0000 7fe3d5b74640 1 Processor -- start 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.115+0000 7fe3d5b74640 1 -- start start 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0102a60 0x7fe3d019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0107fd0 0x7fe3d019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111dd0 con 0x7fe3d0103410 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111c50 con 0x7fe3d0107fd0 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3d5b74640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111f50 con 0x7fe3d0102a60 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34392/0 (socket says 172.21.15.62:34392) 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 -- 172.21.15.62:0/2774747675 learned_addr learned my addr 172.21.15.62:0/2774747675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3cf7fe640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0102a60 0x7fe3d019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3cffff640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0107fd0 0x7fe3d019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 -- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0102a60 msgr2=0x7fe3d019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0102a60 0x7fe3d019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 -- 172.21.15.62:0/2774747675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0107fd0 msgr2=0x7fe3d019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3d0107fd0 0x7fe3d019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3ceffd640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d019f4d0 con 0x7fe3d0103410 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.116+0000 7fe3cf7fe640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3d0102a60 0x7fe3d019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:18.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3ceffd640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d019aa60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8005940 tx=0x7fe3b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b803f030 con 0x7fe3d0103410 2024-08-16T19:09:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b8003ce0 con 0x7fe3d0103410 2024-08-16T19:09:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8035070 con 0x7fe3d0103410 2024-08-16T19:09:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d019f760 con 0x7fe3d0103410 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.117+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d019fbf0 con 0x7fe3d0103410 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.118+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3d010e040 con 0x7fe3d0103410 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.119+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe3b8042030 con 0x7fe3d0103410 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3ccff9640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3bc0a2c10 0x7fe3bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3cf7fe640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3bc0a2c10 0x7fe3bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:18.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fe3b80f0100 con 0x7fe3d0103410 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3ccff9640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] conn(0x7fe3bc0adde0 0x7fe3bc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3b8032a80 con 0x7fe3bc0adde0 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe3b80f0550 con 0x7fe3d0103410 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.120+0000 7fe3cf7fe640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3bc0a2c10 0x7fe3bc0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00058c0 tx=0x7fe3c0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.121+0000 7fe3cffff640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] conn(0x7fe3bc0adde0 0x7fe3bc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.121+0000 7fe3cffff640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] conn(0x7fe3bc0adde0 0x7fe3bc0b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:18.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.122+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== osd.8 v2:172.21.15.161:6816/2842601651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3b8032a80 con 0x7fe3bc0adde0 2024-08-16T19:09:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 --> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3d006a580 con 0x7fe3bc0adde0 2024-08-16T19:09:18.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3ccff9640 1 -- 172.21.15.62:0/2774747675 <== osd.8 v2:172.21.15.161:6816/2842601651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe3d006a580 con 0x7fe3bc0adde0 2024-08-16T19:09:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] conn(0x7fe3bc0adde0 msgr2=0x7fe3bc0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] conn(0x7fe3bc0adde0 0x7fe3bc0b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3bc0a2c10 msgr2=0x7fe3bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe3bc0a2c10 0x7fe3bc0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00058c0 tx=0x7fe3c0005ec0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 msgr2=0x7fe3d019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3d5b74640 1 --2- 172.21.15.62:0/2774747675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3d0103410 0x7fe3d019aa60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8005940 tx=0x7fe3b80042f0 comp rx=0 tx=0).stop 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.136+0000 7fe3cffff640 1 -- 172.21.15.62:0/2774747675 reap_dead start 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.137+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 shutdown_connections 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.137+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 >> 172.21.15.62:0/2774747675 conn(0x7fe3d00fc500 msgr2=0x7fe3d00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.137+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 shutdown_connections 2024-08-16T19:09:18.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.137+0000 7fe3d5b74640 1 -- 172.21.15.62:0/2774747675 wait complete. 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- 172.21.15.62:0/3205140236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47541018b0 msgr2=0x7f4754101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 --2- 172.21.15.62:0/3205140236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47541018b0 0x7f4754101d10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4744009930 tx=0x7f474402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- 172.21.15.62:0/3205140236 shutdown_connections 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 --2- 172.21.15.62:0/3205140236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4754102250 0x7f475410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 --2- 172.21.15.62:0/3205140236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47541018b0 0x7f4754101d10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 --2- 172.21.15.62:0/3205140236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 0x7f4754107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- 172.21.15.62:0/3205140236 >> 172.21.15.62:0/3205140236 conn(0x7f47540fd5f0 msgr2=0x7f47540ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- 172.21.15.62:0/3205140236 shutdown_connections 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- 172.21.15.62:0/3205140236 wait complete. 2024-08-16T19:09:19.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 Processor -- start 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.835+0000 7f475c574640 1 -- start start 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47541018b0 0x7f47541981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 0x7f4754198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 0x7f475419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4754111cc0 con 0x7f4754102250 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4754111b40 con 0x7f47541078b0 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475c574640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4754111e40 con 0x7f47541018b0 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 0x7f4754198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 0x7f4754198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34398/0 (socket says 172.21.15.62:34398) 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 -- 172.21.15.62:0/3562892780 learned_addr learned my addr 172.21.15.62:0/3562892780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475ab11640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 0x7f475419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475a310640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47541018b0 0x7f47541981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 -- 172.21.15.62:0/3562892780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47541018b0 msgr2=0x7f47541981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47541018b0 0x7f47541981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 -- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 msgr2=0x7f475419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 0x7f475419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475419d170 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f475ab11640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47541078b0 0x7f475419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.836+0000 7f4759b0f640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 0x7f4754198700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f474402a940 tx=0x7f4744003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474403f070 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475419d400 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4744031c90 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744035030 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541a4d40 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.837+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47540626b0 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.839+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4744042070 con 0x7f4754102250 2024-08-16T19:09:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.839+0000 7f474b7fe640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47400a2c10 0x7f47400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f475a310640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47400a2c10 0x7f47400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f47440f0450 con 0x7f4754102250 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f474b7fe640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] conn(0x7f47400adde0 0x7f47400b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4744038e90 con 0x7f47400adde0 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f47440f08a0 con 0x7f4754102250 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f475a310640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47400a2c10 0x7f47400a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4750005fd0 tx=0x7f4750005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.840+0000 7f475ab11640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] conn(0x7f47400adde0 0x7f47400b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.841+0000 7f475ab11640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] conn(0x7f47400adde0 0x7f47400b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.842+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== osd.10 v2:172.21.15.155:6824/1300242753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4744038e90 con 0x7f47400adde0 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.855+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 --> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4754062b30 con 0x7f47400adde0 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f474b7fe640 1 -- 172.21.15.62:0/3562892780 <== osd.10 v2:172.21.15.155:6824/1300242753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4754062b30 con 0x7f47400adde0 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] conn(0x7f47400adde0 msgr2=0x7f47400b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] conn(0x7f47400adde0 0x7f47400b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47400a2c10 msgr2=0x7f47400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47400a2c10 0x7f47400a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4750005fd0 tx=0x7f4750005850 comp rx=0 tx=0).stop 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 msgr2=0x7f4754198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 --2- 172.21.15.62:0/3562892780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4754102250 0x7f4754198700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f474402a940 tx=0x7f4744003b90 comp rx=0 tx=0).stop 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475ab11640 1 -- 172.21.15.62:0/3562892780 reap_dead start 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 shutdown_connections 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 >> 172.21.15.62:0/3562892780 conn(0x7f47540fd5f0 msgr2=0x7f47540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:19.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 shutdown_connections 2024-08-16T19:09:19.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:18.856+0000 7f475c574640 1 -- 172.21.15.62:0/3562892780 wait complete. 2024-08-16T19:09:19.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:19 smithi062 ceph-mon[32993]: pgmap v300: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-16T19:09:19.815 INFO:teuthology.orchestra.run.smithi062.stdout:236223201321 2024-08-16T19:09:19.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-16T19:09:19.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:19 smithi155 ceph-mon[37413]: pgmap v300: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-16T19:09:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:19 smithi161 ceph-mon[37576]: pgmap v300: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-16T19:09:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 -- 172.21.15.62:0/1858693071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a81022a0 msgr2=0x7f51a81026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 --2- 172.21.15.62:0/1858693071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a81022a0 0x7f51a81026e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5190009900 tx=0x7f519002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 -- 172.21.15.62:0/1858693071 shutdown_connections 2024-08-16T19:09:20.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 --2- 172.21.15.62:0/1858693071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a8102c20 0x7f51a810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 --2- 172.21.15.62:0/1858693071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a81022a0 0x7f51a81026e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 --2- 172.21.15.62:0/1858693071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51a8100b50 0x7f51a8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.005+0000 7f51adf00640 1 -- 172.21.15.62:0/1858693071 >> 172.21.15.62:0/1858693071 conn(0x7f51a80fc500 msgr2=0x7f51a80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.006+0000 7f51adf00640 1 -- 172.21.15.62:0/1858693071 shutdown_connections 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.006+0000 7f51adf00640 1 -- 172.21.15.62:0/1858693071 wait complete. 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.006+0000 7f51adf00640 1 Processor -- start 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.006+0000 7f51adf00640 1 -- start start 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51a8100b50 0x7f51a819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a81022a0 0x7f51a819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 0x7f51a819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110cb0 con 0x7f51a8102c20 2024-08-16T19:09:20.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110b30 con 0x7f51a81022a0 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51adf00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8110e30 con 0x7f51a8100b50 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 0x7f51a819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 0x7f51a819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34426/0 (socket says 172.21.15.62:34426) 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 -- 172.21.15.62:0/3189858293 learned_addr learned my addr 172.21.15.62:0/3189858293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a77fe640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51a8100b50 0x7f51a819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f519effd640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a81022a0 0x7f51a819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 -- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51a8100b50 msgr2=0x7f51a819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51a8100b50 0x7f51a819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 -- 172.21.15.62:0/3189858293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a81022a0 msgr2=0x7f51a819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a81022a0 0x7f51a819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a819f480 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f519effd640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a81022a0 0x7f51a819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a7fff640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 0x7f51a819eda0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009fd0 tx=0x7f51a000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.007+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0017070 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.008+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a819f770 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.008+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51a0003e50 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.008+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00106c0 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.008+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81a7050 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.009+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51a806a840 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.010+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f51a0003970 con 0x7f51a8102c20 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a57fa640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51940a2c10 0x7f51940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a77fe640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51940a2c10 0x7f51940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f51a00cf650 con 0x7f51a8102c20 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a57fa640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] conn(0x7f51940adde0 0x7f51940b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51a001de50 con 0x7f51940adde0 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f51a00cfaa0 con 0x7f51a8102c20 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f51a77fe640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51940a2c10 0x7f51940a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5198005fd0 tx=0x7f519800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.011+0000 7f519effd640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] conn(0x7f51940adde0 0x7f51940b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.012+0000 7f519effd640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] conn(0x7f51940adde0 0x7f51940b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.013+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== osd.5 v2:172.21.15.161:6808/2837279313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51a001de50 con 0x7f51940adde0 2024-08-16T19:09:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.031+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 --> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51a8101ee0 con 0x7f51940adde0 2024-08-16T19:09:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.031+0000 7f51a57fa640 1 -- 172.21.15.62:0/3189858293 <== osd.5 v2:172.21.15.161:6808/2837279313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f51a8101ee0 con 0x7f51940adde0 2024-08-16T19:09:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.032+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] conn(0x7f51940adde0 msgr2=0x7f51940b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.032+0000 7f51adf00640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] conn(0x7f51940adde0 0x7f51940b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51940a2c10 msgr2=0x7f51940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51940a2c10 0x7f51940a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5198005fd0 tx=0x7f519800b390 comp rx=0 tx=0).stop 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 msgr2=0x7f51a819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 --2- 172.21.15.62:0/3189858293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51a8102c20 0x7f51a819eda0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009fd0 tx=0x7f51a000b260 comp rx=0 tx=0).stop 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51a7fff640 1 -- 172.21.15.62:0/3189858293 reap_dead start 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 shutdown_connections 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 >> 172.21.15.62:0/3189858293 conn(0x7f51a80fc500 msgr2=0x7f51a8069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 shutdown_connections 2024-08-16T19:09:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.033+0000 7f51adf00640 1 -- 172.21.15.62:0/3189858293 wait complete. 2024-08-16T19:09:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 -- 172.21.15.62:0/783395185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 msgr2=0x7f65b81026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 --2- 172.21.15.62:0/783395185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b81026e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009900 tx=0x7f65a002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 -- 172.21.15.62:0/783395185 shutdown_connections 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 --2- 172.21.15.62:0/783395185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b8102c20 0x7f65b810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 --2- 172.21.15.62:0/783395185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b81026e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 --2- 172.21.15.62:0/783395185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8100b50 0x7f65b8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 -- 172.21.15.62:0/783395185 >> 172.21.15.62:0/783395185 conn(0x7f65b80fc500 msgr2=0x7f65b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 -- 172.21.15.62:0/783395185 shutdown_connections 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.520+0000 7f65beebf640 1 -- 172.21.15.62:0/783395185 wait complete. 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 Processor -- start 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 -- start start 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b8100b50 0x7f65b8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8102c20 0x7f65b819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8110cb0 con 0x7f65b81022a0 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8110b30 con 0x7f65b8102c20 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65beebf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8110e30 con 0x7f65b8100b50 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34468/0 (socket says 172.21.15.62:34468) 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 -- 172.21.15.62:0/3111512160 learned_addr learned my addr 172.21.15.62:0/3111512160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65bd45c640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8102c20 0x7f65b819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65bcc5b640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b8100b50 0x7f65b8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 -- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b8100b50 msgr2=0x7f65b8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65b8100b50 0x7f65b8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 -- 172.21.15.62:0/3111512160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8102c20 msgr2=0x7f65b819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8102c20 0x7f65b819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65b7fff640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b819d1c0 con 0x7f65b81022a0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.521+0000 7f65bd45c640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b8102c20 0x7f65b819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65b7fff640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b8198750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f65a0006010 tx=0x7f65a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a003f070 con 0x7f65b81022a0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b819d450 con 0x7f65b81022a0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a0031c30 con 0x7f65b81022a0 2024-08-16T19:09:20.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0035030 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.522+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a4d90 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.523+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65b80626b0 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.524+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65a0042070 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65b5ffb640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f65a40a2c10 0x7f65a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f65a00f0690 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65bcc5b640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f65a40a2c10 0x7f65a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65b5ffb640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] conn(0x7f65a40adde0 0x7f65a40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65a40b0950 con 0x7f65a40adde0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f65a00f0ae0 con 0x7f65b81022a0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.525+0000 7f65bcc5b640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f65a40a2c10 0x7f65a40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005fd0 tx=0x7f65a8009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.526+0000 7f65bd45c640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] conn(0x7f65a40adde0 0x7f65a40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.526+0000 7f65bd45c640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] conn(0x7f65a40adde0 0x7f65a40b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.527+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== osd.11 v2:172.21.15.161:6824/1433195011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65a40b0950 con 0x7f65a40adde0 2024-08-16T19:09:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.541+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 --> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65b8062ab0 con 0x7f65a40adde0 2024-08-16T19:09:20.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.541+0000 7f65b5ffb640 1 -- 172.21.15.62:0/3111512160 <== osd.11 v2:172.21.15.161:6824/1433195011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65b8062ab0 con 0x7f65a40adde0 2024-08-16T19:09:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.541+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] conn(0x7f65a40adde0 msgr2=0x7f65a40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.541+0000 7f65beebf640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] conn(0x7f65a40adde0 0x7f65a40b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f65a40a2c10 msgr2=0x7f65a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f65a40a2c10 0x7f65a40a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65a8005fd0 tx=0x7f65a8009870 comp rx=0 tx=0).stop 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 msgr2=0x7f65b8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 --2- 172.21.15.62:0/3111512160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b81022a0 0x7f65b8198750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f65a0006010 tx=0x7f65a0003c10 comp rx=0 tx=0).stop 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65bd45c640 1 -- 172.21.15.62:0/3111512160 reap_dead start 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 shutdown_connections 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 >> 172.21.15.62:0/3111512160 conn(0x7f65b80fc500 msgr2=0x7f65b8069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 shutdown_connections 2024-08-16T19:09:20.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.542+0000 7f65beebf640 1 -- 172.21.15.62:0/3111512160 wait complete. 2024-08-16T19:09:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 -- 172.21.15.62:0/836460085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340102aa0 msgr2=0x7fa340102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 --2- 172.21.15.62:0/836460085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340102aa0 0x7fa340102e80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa330009930 tx=0x7fa33002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 -- 172.21.15.62:0/836460085 shutdown_connections 2024-08-16T19:09:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 --2- 172.21.15.62:0/836460085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa340107fd0 0x7fa34010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 --2- 172.21.15.62:0/836460085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa340103450 0x7fa340107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 --2- 172.21.15.62:0/836460085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340102aa0 0x7fa340102e80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 -- 172.21.15.62:0/836460085 >> 172.21.15.62:0/836460085 conn(0x7fa3400fc500 msgr2=0x7fa3400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 -- 172.21.15.62:0/836460085 shutdown_connections 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.442+0000 7fa3451dd640 1 -- 172.21.15.62:0/836460085 wait complete. 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 Processor -- start 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 -- start start 2024-08-16T19:09:20.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa340102aa0 0x7fa34019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa340103450 0x7fa34019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 0x7fa34019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa340111df0 con 0x7fa340107fd0 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa340111c70 con 0x7fa340103450 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa3451dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa340111f70 con 0x7fa340102aa0 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa33f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 0x7fa34019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa33f59e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 0x7fa34019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34442/0 (socket says 172.21.15.62:34442) 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa33f59e640 1 -- 172.21.15.62:0/3164653581 learned_addr learned my addr 172.21.15.62:0/3164653581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.443+0000 7fa33e59c640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa340103450 0x7fa34019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33ed9d640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa340102aa0 0x7fa34019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 -- 172.21.15.62:0/3164653581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa340102aa0 msgr2=0x7fa34019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa340102aa0 0x7fa34019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 -- 172.21.15.62:0/3164653581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa340103450 msgr2=0x7fa34019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa340103450 0x7fa34019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34019f460 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa33f59e640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 0x7fa34019ed80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa33800eaa0 tx=0x7fa33800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800cdb0 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa338005020 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34019f750 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3380116b0 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.444+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3401a7030 con 0x7fa340107fd0 2024-08-16T19:09:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.445+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3400626b0 con 0x7fa340107fd0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.446+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa338005560 con 0x7fa340107fd0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa327fff640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa3280a2c10 0x7fa3280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa33ed9d640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa3280a2c10 0x7fa3280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa3380cf710 con 0x7fa340107fd0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa327fff640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] conn(0x7fa3280adde0 0x7fa3280b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa338018ec0 con 0x7fa3280adde0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa3380d6050 con 0x7fa340107fd0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.447+0000 7fa33e59c640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] conn(0x7fa3280adde0 0x7fa3280b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.448+0000 7fa33ed9d640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa3280a2c10 0x7fa3280a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa330000c00 tx=0x7fa330005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.448+0000 7fa33e59c640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] conn(0x7fa3280adde0 0x7fa3280b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.449+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== osd.6 v2:172.21.15.62:6818/3718256322 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa338018ec0 con 0x7fa3280adde0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.462+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 --> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa340062b30 con 0x7fa3280adde0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa327fff640 1 -- 172.21.15.62:0/3164653581 <== osd.6 v2:172.21.15.62:6818/3718256322 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa340062b30 con 0x7fa3280adde0 2024-08-16T19:09:20.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] conn(0x7fa3280adde0 msgr2=0x7fa3280b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] conn(0x7fa3280adde0 0x7fa3280b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa3280a2c10 msgr2=0x7fa3280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fa3280a2c10 0x7fa3280a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa330000c00 tx=0x7fa330005a20 comp rx=0 tx=0).stop 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 msgr2=0x7fa34019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 --2- 172.21.15.62:0/3164653581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa340107fd0 0x7fa34019ed80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa33800eaa0 tx=0x7fa33800ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa33f59e640 1 -- 172.21.15.62:0/3164653581 reap_dead start 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 shutdown_connections 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.463+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 >> 172.21.15.62:0/3164653581 conn(0x7fa3400fc500 msgr2=0x7fa340100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.464+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 shutdown_connections 2024-08-16T19:09:20.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:20.464+0000 7fa3451dd640 1 -- 172.21.15.62:0/3164653581 wait complete. 2024-08-16T19:09:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- 172.21.15.62:0/803952726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1018b0 msgr2=0x7f3bcc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 --2- 172.21.15.62:0/803952726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1018b0 0x7f3bcc101d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009930 tx=0x7f3bb402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- 172.21.15.62:0/803952726 shutdown_connections 2024-08-16T19:09:21.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 --2- 172.21.15.62:0/803952726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bcc102250 0x7f3bcc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 --2- 172.21.15.62:0/803952726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1018b0 0x7f3bcc101d10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 --2- 172.21.15.62:0/803952726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- 172.21.15.62:0/803952726 >> 172.21.15.62:0/803952726 conn(0x7f3bcc0fd5f0 msgr2=0x7f3bcc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- 172.21.15.62:0/803952726 shutdown_connections 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- 172.21.15.62:0/803952726 wait complete. 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 Processor -- start 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.247+0000 7f3bd240a640 1 -- start start 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bcc1018b0 0x7f3bcc074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bcc102250 0x7f3bcc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111d00 con 0x7f3bcc1078b0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111b80 con 0x7f3bcc1018b0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd240a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc111e80 con 0x7f3bcc102250 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50216/0 (socket says 172.21.15.62:50216) 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 -- 172.21.15.62:0/5909981 learned_addr learned my addr 172.21.15.62:0/5909981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bcbfff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bcc1018b0 0x7f3bcc074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bc3fff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bcc102250 0x7f3bcc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 -- 172.21.15.62:0/5909981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bcc102250 msgr2=0x7f3bcc074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bcc102250 0x7f3bcc074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 -- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bcc1018b0 msgr2=0x7f3bcc074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bcc1018b0 0x7f3bcc074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc1a6680 con 0x7f3bcc1078b0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bd09a7640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc074ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400eaa0 tx=0x7f3bc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.248+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400cdb0 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.249+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bc4005020 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.249+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1a6970 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.249+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40116b0 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.249+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a6e30 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.249+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bcc062630 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.251+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3bc4005560 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.251+0000 7f3bc9ffb640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bb80a2c10 0x7f3bb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.251+0000 7f3bcbfff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bb80a2c10 0x7f3bb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3bc40cfae0 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc9ffb640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] conn(0x7f3bb80adde0 0x7f3bb80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bc4014ec0 con 0x7f3bb80adde0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3bc40cff30 con 0x7f3bcc1078b0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bcbfff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bb80a2c10 0x7f3bb80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00bb60 tx=0x7f3bbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc3fff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] conn(0x7f3bb80adde0 0x7f3bb80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.252+0000 7f3bc3fff640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] conn(0x7f3bb80adde0 0x7f3bb80b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.253+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== osd.1 v2:172.21.15.155:6800/471904740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bc4014ec0 con 0x7f3bb80adde0 2024-08-16T19:09:21.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.267+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 --> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bcc062ab0 con 0x7f3bb80adde0 2024-08-16T19:09:21.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bc9ffb640 1 -- 172.21.15.62:0/5909981 <== osd.1 v2:172.21.15.155:6800/471904740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3bcc062ab0 con 0x7f3bb80adde0 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] conn(0x7f3bb80adde0 msgr2=0x7f3bb80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] conn(0x7f3bb80adde0 0x7f3bb80b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bb80a2c10 msgr2=0x7f3bb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3bb80a2c10 0x7f3bb80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00bb60 tx=0x7f3bbc00b390 comp rx=0 tx=0).stop 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 msgr2=0x7f3bcc074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 --2- 172.21.15.62:0/5909981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3bcc1078b0 0x7f3bcc074ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400eaa0 tx=0x7f3bc400ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd09a7640 1 -- 172.21.15.62:0/5909981 reap_dead start 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 shutdown_connections 2024-08-16T19:09:21.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 >> 172.21.15.62:0/5909981 conn(0x7f3bcc0fd5f0 msgr2=0x7f3bcc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:21.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 shutdown_connections 2024-08-16T19:09:21.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.268+0000 7f3bd240a640 1 -- 172.21.15.62:0/5909981 wait complete. 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 -- 172.21.15.62:0/1606442633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55681018c0 msgr2=0x7f5568101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 --2- 172.21.15.62:0/1606442633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55681018c0 0x7f5568101d20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f555c009930 tx=0x7f555c02a420 comp rx=0 tx=0).stop 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 -- 172.21.15.62:0/1606442633 shutdown_connections 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 --2- 172.21.15.62:0/1606442633 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5568102260 0x7f556810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 --2- 172.21.15.62:0/1606442633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55681018c0 0x7f5568101d20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 --2- 172.21.15.62:0/1606442633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681078c0 0x7f5568107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 -- 172.21.15.62:0/1606442633 >> 172.21.15.62:0/1606442633 conn(0x7f55680fd5e0 msgr2=0x7f55680ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 -- 172.21.15.62:0/1606442633 shutdown_connections 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.440+0000 7f556e85e640 1 -- 172.21.15.62:0/1606442633 wait complete. 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 Processor -- start 2024-08-16T19:09:21.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 -- start start 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681018c0 0x7f55681981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 0x7f55681986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55681078c0 0x7f556819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111cc0 con 0x7f5568102260 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111b40 con 0x7f55681018c0 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556e85e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111e40 con 0x7f55681078c0 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f55677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 0x7f55681986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f55677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 0x7f55681986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50232/0 (socket says 172.21.15.62:50232) 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f55677fe640 1 -- 172.21.15.62:0/3832381879 learned_addr learned my addr 172.21.15.62:0/3832381879 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f556cdfb640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55681078c0 0x7f556819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.441+0000 7f5567fff640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681018c0 0x7f55681981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 -- 172.21.15.62:0/3832381879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55681078c0 msgr2=0x7f556819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55681078c0 0x7f556819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 -- 172.21.15.62:0/3832381879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681018c0 msgr2=0x7f55681981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681018c0 0x7f55681981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819d150 con 0x7f5568102260 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f5567fff640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55681018c0 0x7f55681981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55677fe640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 0x7f55681986e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f555c009230 tx=0x7f555c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c03f030 con 0x7f5568102260 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c003ad0 con 0x7f5568102260 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819d3e0 con 0x7f5568102260 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c035070 con 0x7f5568102260 2024-08-16T19:09:21.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.442+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a4d20 con 0x7f5568102260 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.443+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55680626b0 con 0x7f5568102260 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.444+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f555c042030 con 0x7f5568102260 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f55657fa640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f55500a2c90 0x7f55500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f555c0efe40 con 0x7f5568102260 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f5567fff640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f55500a2c90 0x7f55500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f55657fa640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] conn(0x7f55500ade60 0x7f55500b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55500b09d0 con 0x7f55500ade60 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f555c0f0290 con 0x7f5568102260 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f556cdfb640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] conn(0x7f55500ade60 0x7f55500b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.445+0000 7f5567fff640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f55500a2c90 0x7f55500a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5558005d90 tx=0x7f5558005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.446+0000 7f556cdfb640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] conn(0x7f55500ade60 0x7f55500b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:21.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.446+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== osd.0 v2:172.21.15.62:6802/3542478147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55500b09d0 con 0x7f55500ade60 2024-08-16T19:09:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.460+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 --> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5568062ab0 con 0x7f55500ade60 2024-08-16T19:09:21.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.460+0000 7f55657fa640 1 -- 172.21.15.62:0/3832381879 <== osd.0 v2:172.21.15.62:6802/3542478147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5568062ab0 con 0x7f55500ade60 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] conn(0x7f55500ade60 msgr2=0x7f55500b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] conn(0x7f55500ade60 0x7f55500b0300 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f55500a2c90 msgr2=0x7f55500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f55500a2c90 0x7f55500a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5558005d90 tx=0x7f5558005d20 comp rx=0 tx=0).stop 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 msgr2=0x7f55681986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 --2- 172.21.15.62:0/3832381879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5568102260 0x7f55681986e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f555c009230 tx=0x7f555c004270 comp rx=0 tx=0).stop 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556cdfb640 1 -- 172.21.15.62:0/3832381879 reap_dead start 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 shutdown_connections 2024-08-16T19:09:21.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 >> 172.21.15.62:0/3832381879 conn(0x7f55680fd5e0 msgr2=0x7f55680fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:21.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 shutdown_connections 2024-08-16T19:09:21.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:21.461+0000 7f556e85e640 1 -- 172.21.15.62:0/3832381879 wait complete. 2024-08-16T19:09:21.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:21 smithi062 ceph-mon[32993]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-16T19:09:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:21 smithi155 ceph-mon[37413]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-16T19:09:21.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:21 smithi161 ceph-mon[37576]: pgmap v301: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-08-16T19:09:22.478 INFO:teuthology.orchestra.run.smithi062.stdout:133143986235 2024-08-16T19:09:22.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-16T19:09:23.142 INFO:teuthology.orchestra.run.smithi062.stdout:154618822711 2024-08-16T19:09:23.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-16T19:09:23.614 INFO:teuthology.orchestra.run.smithi062.stdout:184683593779 2024-08-16T19:09:23.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-16T19:09:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:23 smithi062 ceph-mon[32993]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:23 smithi155 ceph-mon[37413]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:23 smithi161 ceph-mon[37576]: pgmap v302: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:24.282 INFO:teuthology.orchestra.run.smithi062.stdout:77309411395 2024-08-16T19:09:24.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 -- 172.21.15.62:0/2834091366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50103410 msgr2=0x7f3b501078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 --2- 172.21.15.62:0/2834091366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50103410 0x7f3b501078c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009930 tx=0x7f3b4002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 -- 172.21.15.62:0/2834091366 shutdown_connections 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 --2- 172.21.15.62:0/2834091366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50107fd0 0x7f3b5010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 --2- 172.21.15.62:0/2834091366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50103410 0x7f3b501078c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 --2- 172.21.15.62:0/2834091366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b50102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 -- 172.21.15.62:0/2834091366 >> 172.21.15.62:0/2834091366 conn(0x7f3b500fc500 msgr2=0x7f3b500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 -- 172.21.15.62:0/2834091366 shutdown_connections 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.613+0000 7f3b57772640 1 -- 172.21.15.62:0/2834091366 wait complete. 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 Processor -- start 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 -- start start 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b501983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50103410 0x7f3b501988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50111da0 con 0x7f3b50107fd0 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50111c20 con 0x7f3b50102a60 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b57772640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3b50111f20 con 0x7f3b50103410 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b55d0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b55d0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50246/0 (socket says 172.21.15.62:50246) 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b55d0f640 1 -- 172.21.15.62:0/3108170302 learned_addr learned my addr 172.21.15.62:0/3108170302 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.614+0000 7f3b54d0d640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50103410 0x7f3b501988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b5550e640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b501983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 -- 172.21.15.62:0/3108170302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50103410 msgr2=0x7f3b501988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50103410 0x7f3b501988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 -- 172.21.15.62:0/3108170302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 msgr2=0x7f3b501983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b501983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 -- 172.21.15.62:0/3108170302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5019d350 con 0x7f3b50107fd0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b5550e640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b501983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b55d0f640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800eaa0 tx=0x7f3b4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4800cdb0 con 0x7f3b50107fd0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5019d640 con 0x7f3b50107fd0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b48005020 con 0x7f3b50107fd0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480116b0 con 0x7f3b50107fd0 2024-08-16T19:09:24.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.615+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501a4f20 con 0x7f3b50107fd0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.616+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b50102e40 con 0x7f3b50107fd0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.617+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3b48005560 con 0x7f3b50107fd0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.618+0000 7f3b467fc640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 0x7f3b3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.618+0000 7f3b5550e640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 0x7f3b3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.618+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f3b480cf740 con 0x7f3b50107fd0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.618+0000 7f3b5550e640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 0x7f3b3c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0058c0 tx=0x7f3b4c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:24.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.622+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3b48016030 con 0x7f3b50107fd0 2024-08-16T19:09:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.803+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3b50109fe0 con 0x7f3b50107fd0 2024-08-16T19:09:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.803+0000 7f3b467fc640 1 -- 172.21.15.62:0/3108170302 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b4809c1e0 con 0x7f3b50107fd0 2024-08-16T19:09:24.804 INFO:teuthology.orchestra.run.smithi062.stdout:270582939686 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 msgr2=0x7f3b3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 0x7f3b3c0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0058c0 tx=0x7f3b4c005ec0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 msgr2=0x7f3b5019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800eaa0 tx=0x7f3b4800ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 shutdown_connections 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b50107fd0 0x7f3b5019cc70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b50103410 0x7f3b501988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f3b3c0a2c10 0x7f3b3c0a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 --2- 172.21.15.62:0/3108170302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b50102a60 0x7f3b501983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.805+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 >> 172.21.15.62:0/3108170302 conn(0x7f3b500fc500 msgr2=0x7f3b501060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.806+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 shutdown_connections 2024-08-16T19:09:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.806+0000 7f3b57772640 1 -- 172.21.15.62:0/3108170302 wait complete. 2024-08-16T19:09:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 -- 172.21.15.62:0/2327584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58102a60 msgr2=0x7fea58102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 --2- 172.21.15.62:0/2327584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58102a60 0x7fea58102e40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea54009930 tx=0x7fea5402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 -- 172.21.15.62:0/2327584 shutdown_connections 2024-08-16T19:09:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 --2- 172.21.15.62:0/2327584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58107fd0 0x7fea5810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 --2- 172.21.15.62:0/2327584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 0x7fea581078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 --2- 172.21.15.62:0/2327584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58102a60 0x7fea58102e40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 -- 172.21.15.62:0/2327584 >> 172.21.15.62:0/2327584 conn(0x7fea580fc500 msgr2=0x7fea580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 -- 172.21.15.62:0/2327584 shutdown_connections 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.839+0000 7fea606d6640 1 -- 172.21.15.62:0/2327584 wait complete. 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 Processor -- start 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 -- start start 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58102a60 0x7fea58198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 0x7fea58198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fea58111dd0 con 0x7fea58107fd0 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fea58111c50 con 0x7fea58102a60 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea606d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fea58111f50 con 0x7fea58103410 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50260/0 (socket says 172.21.15.62:50260) 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 -- 172.21.15.62:0/2748309832 learned_addr learned my addr 172.21.15.62:0/2748309832 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5dc71640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 0x7fea58198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5e472640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58102a60 0x7fea58198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 -- 172.21.15.62:0/2748309832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 msgr2=0x7fea58198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 0x7fea58198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 -- 172.21.15.62:0/2748309832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58102a60 msgr2=0x7fea58198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58102a60 0x7fea58198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.840+0000 7fea5ec73640 1 -- 172.21.15.62:0/2748309832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819d300 con 0x7fea58107fd0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea5ec73640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea5000eb10 tx=0x7fea5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50004b20 con 0x7fea58107fd0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea50004cc0 con 0x7fea58107fd0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50010760 con 0x7fea58107fd0 2024-08-16T19:09:24.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5819d5f0 con 0x7fea58107fd0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.841+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a4ed0 con 0x7fea58107fd0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.842+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5806a580 con 0x7fea58107fd0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.843+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea50005020 con 0x7fea58107fd0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.844+0000 7fea4f7fe640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 0x7fea440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.844+0000 7fea5e472640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 0x7fea440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.844+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fea500cfa70 con 0x7fea58107fd0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.844+0000 7fea5e472640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 0x7fea440a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea54000c00 tx=0x7fea54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:24.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:24.848+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fea5009c510 con 0x7fea58107fd0 2024-08-16T19:09:25.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.025+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fea58109fe0 con 0x7fea58107fd0 2024-08-16T19:09:25.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.025+0000 7fea4f7fe640 1 -- 172.21.15.62:0/2748309832 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea5009be10 con 0x7fea58107fd0 2024-08-16T19:09:25.026 INFO:teuthology.orchestra.run.smithi062.stdout:103079215169 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 msgr2=0x7fea440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 0x7fea440a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea54000c00 tx=0x7fea54005a20 comp rx=0 tx=0).stop 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 msgr2=0x7fea5819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea5000eb10 tx=0x7fea5000c2d0 comp rx=0 tx=0).stop 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 shutdown_connections 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea58107fd0 0x7fea5819cc20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea58103410 0x7fea58198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:25.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fea440a2c90 0x7fea440a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:25.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 --2- 172.21.15.62:0/2748309832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea58102a60 0x7fea58198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:25.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.027+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 >> 172.21.15.62:0/2748309832 conn(0x7fea580fc500 msgr2=0x7fea58105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:25.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.028+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 shutdown_connections 2024-08-16T19:09:25.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:25.028+0000 7fea606d6640 1 -- 172.21.15.62:0/2748309832 wait complete. 2024-08-16T19:09:25.694 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:25 smithi062 ceph-mon[32993]: pgmap v303: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:25.694 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3108170302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-16T19:09:25.694 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2748309832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-16T19:09:25.727 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215169 for osd.3 2024-08-16T19:09:25.727 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:25 smithi155 ceph-mon[37413]: pgmap v303: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:25 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3108170302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-16T19:09:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:25 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2748309832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-16T19:09:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:25 smithi161 ceph-mon[37576]: pgmap v303: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-08-16T19:09:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:25 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3108170302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-16T19:09:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:25 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2748309832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-16T19:09:26.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939684 got 270582939686 for osd.9 2024-08-16T19:09:26.110 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:26.610 INFO:teuthology.orchestra.run.smithi062.stdout:34359738444 2024-08-16T19:09:26.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-16T19:09:27.241 INFO:teuthology.orchestra.run.smithi062.stdout:55834574919 2024-08-16T19:09:27.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-16T19:09:27.660 INFO:teuthology.orchestra.run.smithi062.stdout:326417514525 2024-08-16T19:09:27.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-16T19:09:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:27 smithi062 ceph-mon[32993]: pgmap v304: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:27.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:27 smithi155 ceph-mon[37413]: pgmap v304: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:27 smithi161 ceph-mon[37576]: pgmap v304: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:28.085 INFO:teuthology.orchestra.run.smithi062.stdout:292057776160 2024-08-16T19:09:28.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-16T19:09:28.152 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.159 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.160 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.207 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.235 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.570 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 -- 172.21.15.62:0/2385309629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec0742d0 msgr2=0x7fd4ec0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/2385309629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec0746b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009900 tx=0x7fd4dc02a410 comp rx=0 tx=0).stop 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 -- 172.21.15.62:0/2385309629 shutdown_connections 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/2385309629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec10e1a0 0x7fd4ec1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/2385309629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec074c80 0x7fd4ec10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/2385309629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec0746b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 -- 172.21.15.62:0/2385309629 >> 172.21.15.62:0/2385309629 conn(0x7fd4ec0fd600 msgr2=0x7fd4ec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 -- 172.21.15.62:0/2385309629 shutdown_connections 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.928+0000 7fd4f23cb640 1 -- 172.21.15.62:0/2385309629 wait complete. 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 Processor -- start 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 -- start start 2024-08-16T19:09:28.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec10e1a0 0x7fd4ec19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec113f30 con 0x7fd4ec074c80 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec113db0 con 0x7fd4ec10e1a0 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f23cb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec1140b0 con 0x7fd4ec0742d0 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50264/0 (socket says 172.21.15.62:50264) 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 -- 172.21.15.62:0/953056747 learned_addr learned my addr 172.21.15.62:0/953056747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4f0968640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec10e1a0 0x7fd4ec19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4ebfff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 -- 172.21.15.62:0/953056747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec0742d0 msgr2=0x7fd4ec19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:28.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 -- 172.21.15.62:0/953056747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec10e1a0 msgr2=0x7fd4ec19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec10e1a0 0x7fd4ec19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.929+0000 7fd4e3fff640 1 -- 172.21.15.62:0/953056747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec19f360 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4e3fff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400d950 tx=0x7fd4d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4017070 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d4009c30 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec19f650 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400f6b0 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.930+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1a6f30 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.931+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec0746b0 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.932+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd4d40109d0 con 0x7fd4ec074c80 2024-08-16T19:09:28.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.933+0000 7fd4e9ffb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 0x7fd4d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:28.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.933+0000 7fd4ebfff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 0x7fd4d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:28.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.933+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd4d40cf070 con 0x7fd4ec074c80 2024-08-16T19:09:28.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.933+0000 7fd4ebfff640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 0x7fd4d80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009230 tx=0x7fd4dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:28.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:28.937+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd4d4009de0 con 0x7fd4ec074c80 2024-08-16T19:09:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.115+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd4ec109320 con 0x7fd4ec074c80 2024-08-16T19:09:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.116+0000 7fd4e9ffb640 1 -- 172.21.15.62:0/953056747 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd4d409b4c0 con 0x7fd4ec074c80 2024-08-16T19:09:29.116 INFO:teuthology.orchestra.run.smithi062.stdout:184683593781 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 msgr2=0x7fd4d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 0x7fd4d80a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009230 tx=0x7fd4dc0058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 msgr2=0x7fd4ec19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400d950 tx=0x7fd4d400de10 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 shutdown_connections 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd4d80a2c90 0x7fd4d80a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4ec10e1a0 0x7fd4ec19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4ec074c80 0x7fd4ec19a8f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 --2- 172.21.15.62:0/953056747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4ec0742d0 0x7fd4ec19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.118+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 >> 172.21.15.62:0/953056747 conn(0x7fd4ec0fd600 msgr2=0x7fd4ec0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.119+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 shutdown_connections 2024-08-16T19:09:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.119+0000 7fd4f23cb640 1 -- 172.21.15.62:0/953056747 wait complete. 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 -- 172.21.15.62:0/3590052100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e01022a0 msgr2=0x7f84e01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 --2- 172.21.15.62:0/3590052100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e01022a0 0x7f84e01026e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02a480 comp rx=0 tx=0).stop 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 -- 172.21.15.62:0/3590052100 shutdown_connections 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 --2- 172.21.15.62:0/3590052100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0102c20 0x7f84e010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 --2- 172.21.15.62:0/3590052100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e01022a0 0x7f84e01026e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 --2- 172.21.15.62:0/3590052100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e0100b50 0x7f84e0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 -- 172.21.15.62:0/3590052100 >> 172.21.15.62:0/3590052100 conn(0x7f84e00fc500 msgr2=0x7f84e00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 -- 172.21.15.62:0/3590052100 shutdown_connections 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.128+0000 7f84e71fe640 1 -- 172.21.15.62:0/3590052100 wait complete. 2024-08-16T19:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 Processor -- start 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 -- start start 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 0x7f84e0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e01022a0 0x7f84e01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0110c70 con 0x7f84e0102c20 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0110af0 con 0x7f84e0100b50 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e71fe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0110df0 con 0x7f84e01022a0 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e579b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e579b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50288/0 (socket says 172.21.15.62:50288) 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e579b640 1 -- 172.21.15.62:0/1348126870 learned_addr learned my addr 172.21.15.62:0/1348126870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.129+0000 7f84e4f9a640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 0x7f84e0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 -- 172.21.15.62:0/1348126870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e01022a0 msgr2=0x7f84e01995c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e01022a0 0x7f84e01995c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 -- 172.21.15.62:0/1348126870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 msgr2=0x7f84e0199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 0x7f84e0199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 -- 172.21.15.62:0/1348126870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01a4640 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e4f9a640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 0x7f84e0199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e579b640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84d800eaa0 tx=0x7f84d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800cdb0 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01a4810 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d8005020 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d80116b0 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.130+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01a4d20 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.131+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e0100f30 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.132+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f84d8005560 con 0x7f84e0102c20 2024-08-16T19:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.133+0000 7f84ddffb640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 0x7f84d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.133+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f84d80cf720 con 0x7f84e0102c20 2024-08-16T19:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.133+0000 7f84e4f9a640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 0x7f84d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.134+0000 7f84e4f9a640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 0x7f84d00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005d00 tx=0x7f84d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.138+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f84d8016030 con 0x7f84e0102c20 2024-08-16T19:09:29.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.319+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f84e0108dc0 con 0x7f84e0102c20 2024-08-16T19:09:29.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.319+0000 7f84ddffb640 1 -- 172.21.15.62:0/1348126870 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f84d809c1c0 con 0x7f84e0102c20 2024-08-16T19:09:29.320 INFO:teuthology.orchestra.run.smithi062.stdout:206158430256 2024-08-16T19:09:29.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 msgr2=0x7f84d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 0x7f84d00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005d00 tx=0x7f84d400b410 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 msgr2=0x7f84e01a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f84d800eaa0 tx=0x7f84d800ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 shutdown_connections 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0102c20 0x7f84e01a4100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84e01022a0 0x7f84e01995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f84d00a2c10 0x7f84d00a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 --2- 172.21.15.62:0/1348126870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84e0100b50 0x7f84e0199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 >> 172.21.15.62:0/1348126870 conn(0x7f84e00fc500 msgr2=0x7f84e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 shutdown_connections 2024-08-16T19:09:29.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:29.321+0000 7f84e71fe640 1 -- 172.21.15.62:0/1348126870 wait complete. 2024-08-16T19:09:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:29 smithi062 ceph-mon[32993]: pgmap v305: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:29 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/953056747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-16T19:09:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:29 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1348126870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-16T19:09:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:29 smithi155 ceph-mon[37413]: pgmap v305: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:29 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/953056747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-16T19:09:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:29 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1348126870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-16T19:09:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:29 smithi161 ceph-mon[37576]: pgmap v305: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:29 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/953056747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-16T19:09:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:29 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1348126870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-16T19:09:30.410 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430253 got 206158430256 for osd.7 2024-08-16T19:09:30.410 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 -- 172.21.15.62:0/1438951126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 msgr2=0x7f7388107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 --2- 172.21.15.62:0/1438951126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f7388107c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7378009930 tx=0x7f737802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 -- 172.21.15.62:0/1438951126 shutdown_connections 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 --2- 172.21.15.62:0/1438951126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 0x7f738810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 --2- 172.21.15.62:0/1438951126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 0x7f7388101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 --2- 172.21.15.62:0/1438951126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f7388107c90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 -- 172.21.15.62:0/1438951126 >> 172.21.15.62:0/1438951126 conn(0x7f73880fd5f0 msgr2=0x7f73880ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 -- 172.21.15.62:0/1438951126 shutdown_connections 2024-08-16T19:09:31.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.432+0000 7f738da8c640 1 -- 172.21.15.62:0/1438951126 wait complete. 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 Processor -- start 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 -- start start 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 0x7f7388198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 0x7f73881986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7388111cb0 con 0x7f73881078b0 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7388111b30 con 0x7f73881018b0 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f738da8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7388111e30 con 0x7f7388102250 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f7387fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f7387fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50598/0 (socket says 172.21.15.62:50598) 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f7387fff640 1 -- 172.21.15.62:0/4069263760 learned_addr learned my addr 172.21.15.62:0/4069263760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f7386ffd640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 0x7f73881986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.433+0000 7f73877fe640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 0x7f7388198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 -- 172.21.15.62:0/4069263760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 msgr2=0x7f73881986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 0x7f73881986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 -- 172.21.15.62:0/4069263760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 msgr2=0x7f7388198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 0x7f7388198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 -- 172.21.15.62:0/4069263760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738819d120 con 0x7f73881078b0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7387fff640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f738000eaa0 tx=0x7f738000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738000cdb0 con 0x7f73881078b0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7380005020 con 0x7f73881078b0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738819d410 con 0x7f73881078b0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800116b0 con 0x7f73881078b0 2024-08-16T19:09:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.434+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73881a4cf0 con 0x7f73881078b0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.435+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7388101d10 con 0x7f73881078b0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.437+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7380005560 con 0x7f73881078b0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.437+0000 7f7384ff9640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 0x7f73700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.438+0000 7f73877fe640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 0x7f73700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.438+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f73800cf980 con 0x7f73881078b0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.438+0000 7f73877fe640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 0x7f73700a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7378009900 tx=0x7f73780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.441+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7380016030 con 0x7f73881078b0 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 -- 172.21.15.62:0/3995367222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4100b50 msgr2=0x7f23c4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 --2- 172.21.15.62:0/3995367222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4100b50 0x7f23c4100f30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009900 tx=0x7f23b402a480 comp rx=0 tx=0).stop 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 -- 172.21.15.62:0/3995367222 shutdown_connections 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 --2- 172.21.15.62:0/3995367222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c4102c20 0x7f23c410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 --2- 172.21.15.62:0/3995367222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c41022a0 0x7f23c41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 --2- 172.21.15.62:0/3995367222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4100b50 0x7f23c4100f30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 -- 172.21.15.62:0/3995367222 >> 172.21.15.62:0/3995367222 conn(0x7f23c40fc500 msgr2=0x7f23c40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.912+0000 7f23c8d16640 1 -- 172.21.15.62:0/3995367222 shutdown_connections 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.913+0000 7f23c8d16640 1 -- 172.21.15.62:0/3995367222 wait complete. 2024-08-16T19:09:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.913+0000 7f23c8d16640 1 Processor -- start 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.913+0000 7f23c8d16640 1 -- start start 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 0x7f23c41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41022a0 0x7f23c4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110cb0 con 0x7f23c4102c20 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110b30 con 0x7f23c41022a0 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c8d16640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110e30 con 0x7f23c4100b50 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 0x7f23c41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50586/0 (socket says 172.21.15.62:50586) 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c2d9d640 1 -- 172.21.15.62:0/2146454967 learned_addr learned my addr 172.21.15.62:0/2146454967 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c1d9b640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41022a0 0x7f23c4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 -- 172.21.15.62:0/2146454967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41022a0 msgr2=0x7f23c4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41022a0 0x7f23c4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 -- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 msgr2=0x7f23c419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c259c640 1 -- 172.21.15.62:0/2146454967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c419d2a0 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.914+0000 7f23c2d9d640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23c259c640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 0x7f23c41982f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005940 tx=0x7f23b4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b403f070 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c419d530 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4031c30 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4035030 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.915+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c41a4e70 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.916+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c4100f30 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.917+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23b4042070 con 0x7f23c4100b50 2024-08-16T19:09:31.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.918+0000 7f23b37fe640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 0x7f23ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.918+0000 7f23c1d9b640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 0x7f23ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.918+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f23b40f04a0 con 0x7f23c4100b50 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.918+0000 7f23c1d9b640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 0x7f23ac0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ffab0 tx=0x7f23b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.921+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f23b40bc8f0 con 0x7f23c4100b50 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.111+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f23c41a5040 con 0x7f23c4100b50 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stdout:77309411397 2024-08-16T19:09:31.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.111+0000 7f23b37fe640 1 -- 172.21.15.62:0/2146454967 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23b40c0bf0 con 0x7f23c4100b50 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 msgr2=0x7f23ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 0x7f23ac0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ffab0 tx=0x7f23b8005f90 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 msgr2=0x7f23c41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 0x7f23c41982f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005940 tx=0x7f23b4003c10 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 shutdown_connections 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23c4102c20 0x7f23c419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f23ac0a2c10 0x7f23ac0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41022a0 0x7f23c4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 --2- 172.21.15.62:0/2146454967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23c4100b50 0x7f23c41982f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 >> 172.21.15.62:0/2146454967 conn(0x7f23c40fc500 msgr2=0x7f23c4069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:31.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 shutdown_connections 2024-08-16T19:09:31.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.113+0000 7f23c8d16640 1 -- 172.21.15.62:0/2146454967 wait complete. 2024-08-16T19:09:31.549 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:31 smithi062 ceph-mon[32993]: pgmap v306: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:31.549 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:31 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4103425590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-16T19:09:31.549 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:31 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2146454967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-16T19:09:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.621+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f73881994d0 con 0x7f73881078b0 2024-08-16T19:09:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.621+0000 7f7384ff9640 1 -- 172.21.15.62:0/4069263760 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73881994d0 con 0x7f73881078b0 2024-08-16T19:09:31.622 INFO:teuthology.orchestra.run.smithi062.stdout:154618822713 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.623+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 msgr2=0x7f73700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.623+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 0x7f73700a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7378009900 tx=0x7f73780058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.623+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 msgr2=0x7f738819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.623+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f738000eaa0 tx=0x7f738000ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 shutdown_connections 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73881078b0 0x7f738819ca40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7388102250 0x7f73881986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f73700a2c90 0x7f73700a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 --2- 172.21.15.62:0/4069263760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73881018b0 0x7f7388198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:31.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 >> 172.21.15.62:0/4069263760 conn(0x7f73880fd5f0 msgr2=0x7f73880fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:31.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 shutdown_connections 2024-08-16T19:09:31.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.624+0000 7f738da8c640 1 -- 172.21.15.62:0/4069263760 wait complete. 2024-08-16T19:09:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:31 smithi155 ceph-mon[37413]: pgmap v306: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:31 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4103425590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-16T19:09:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:31 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2146454967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-16T19:09:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:31 smithi161 ceph-mon[37576]: pgmap v306: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:31 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4103425590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-16T19:09:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:31 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2146454967' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-16T19:09:32.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411397 for osd.2 2024-08-16T19:09:32.182 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 -- 172.21.15.62:0/3139604301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 msgr2=0x7f4e4c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 --2- 172.21.15.62:0/3139604301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c10b880 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009900 tx=0x7f4e4002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 -- 172.21.15.62:0/3139604301 shutdown_connections 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 --2- 172.21.15.62:0/3139604301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 --2- 172.21.15.62:0/3139604301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c10b880 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 --2- 172.21.15.62:0/3139604301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c102a70 0x7f4e4c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 -- 172.21.15.62:0/3139604301 >> 172.21.15.62:0/3139604301 conn(0x7f4e4c0fc4b0 msgr2=0x7f4e4c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 -- 172.21.15.62:0/3139604301 shutdown_connections 2024-08-16T19:09:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.818+0000 7f4e51a33640 1 -- 172.21.15.62:0/3139604301 wait complete. 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 Processor -- start 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 -- start start 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c102a70 0x7f4e4c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c111d00 con 0x7f4e4c103390 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c111b80 con 0x7f4e4c102a70 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e51a33640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c111e80 con 0x7f4e4c10bea0 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e4a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e4a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50312/0 (socket says 172.21.15.62:50312) 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.819+0000 7f4e4a7fc640 1 -- 172.21.15.62:0/4103425590 learned_addr learned my addr 172.21.15.62:0/4103425590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4b7fe640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4affd640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c102a70 0x7f4e4c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 -- 172.21.15.62:0/4103425590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 msgr2=0x7f4e4c101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 -- 172.21.15.62:0/4103425590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c102a70 msgr2=0x7f4e4c1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c102a70 0x7f4e4c1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stdout:236223201323 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 -- 172.21.15.62:0/4103425590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c1a67a0 con 0x7f4e4c103390 2024-08-16T19:09:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4b7fe640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e4a7fc640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006010 tx=0x7f4e400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4003f030 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e40003ce0 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1a6a30 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40035070 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.820+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1a6f60 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.821+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c102e50 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.823+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4e40042030 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.824+0000 7f4e50a31640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 0x7f4e340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.824+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f4e400efd50 con 0x7f4e4c103390 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.824+0000 7f4e4affd640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 0x7f4e340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.825+0000 7f4e4affd640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 0x7f4e340a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c005fd0 tx=0x7f4e3c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:30.827+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4e400bc8b0 con 0x7f4e4c103390 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.011+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f4e4c106f10 con 0x7f4e4c103390 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.012+0000 7f4e50a31640 1 -- 172.21.15.62:0/4103425590 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4e400bc1b0 con 0x7f4e4c103390 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 msgr2=0x7f4e340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 0x7f4e340a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c005fd0 tx=0x7f4e3c005850 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 msgr2=0x7f4e4c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e40006010 tx=0x7f4e400042f0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 shutdown_connections 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e4c10bea0 0x7f4e4c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e4c103390 0x7f4e4c100c10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4e340a2c90 0x7f4e340a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 --2- 172.21.15.62:0/4103425590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e4c102a70 0x7f4e4c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 >> 172.21.15.62:0/4103425590 conn(0x7f4e4c0fc4b0 msgr2=0x7f4e4c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 shutdown_connections 2024-08-16T19:09:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:31.014+0000 7f4e51a33640 1 -- 172.21.15.62:0/4103425590 wait complete. 2024-08-16T19:09:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:32 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4069263760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-16T19:09:32.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:32 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4069263760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-16T19:09:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:32 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4069263760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 -- 172.21.15.62:0/3090881523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1007b0 msgr2=0x7f2aec100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 --2- 172.21.15.62:0/3090881523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1007b0 0x7f2aec100c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009930 tx=0x7f2ae002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 -- 172.21.15.62:0/3090881523 shutdown_connections 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 --2- 172.21.15.62:0/3090881523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 0x7f2aec10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 --2- 172.21.15.62:0/3090881523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1007b0 0x7f2aec100c10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 --2- 172.21.15.62:0/3090881523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1067b0 0x7f2aec106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 -- 172.21.15.62:0/3090881523 >> 172.21.15.62:0/3090881523 conn(0x7f2aec0fc4d0 msgr2=0x7f2aec0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 -- 172.21.15.62:0/3090881523 shutdown_connections 2024-08-16T19:09:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.898+0000 7f2af4348640 1 -- 172.21.15.62:0/3090881523 wait complete. 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 Processor -- start 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 -- start start 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 0x7f2aec1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 0x7f2aec197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110b70 con 0x7f2aec1067b0 2024-08-16T19:09:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1109f0 con 0x7f2aec1007b0 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.899+0000 7f2af4348640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110cf0 con 0x7f2aec101150 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af28e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af28e5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50626/0 (socket says 172.21.15.62:50626) 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af28e5640 1 -- 172.21.15.62:0/3835994296 learned_addr learned my addr 172.21.15.62:0/3835994296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 0x7f2aec1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af18e3640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 0x7f2aec197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 -- 172.21.15.62:0/3835994296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 msgr2=0x7f2aec197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 0x7f2aec197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 -- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 msgr2=0x7f2aec19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 -- 172.21.15.62:0/3835994296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec19c080 con 0x7f2aec1007b0 2024-08-16T19:09:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af28e5640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.900+0000 7f2af20e4640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 0x7f2aec1970d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00eb00 tx=0x7f2adc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.901+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc010430 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.901+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec19c370 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.901+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2adc010ab0 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.901+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc017bb0 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.901+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec06a4c0 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.902+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec100c10 con 0x7f2aec1007b0 2024-08-16T19:09:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.903+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2adc010c60 con 0x7f2aec1007b0 2024-08-16T19:09:32.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.904+0000 7f2aeb7fe640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 0x7f2ad40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:32.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.904+0000 7f2af18e3640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 0x7f2ad40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:32.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.904+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2adc0d0020 con 0x7f2aec1007b0 2024-08-16T19:09:32.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.905+0000 7f2af18e3640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 0x7f2ad40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009900 tx=0x7f2ae00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:32.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:32.907+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2adc053df0 con 0x7f2aec1007b0 2024-08-16T19:09:33.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.089+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2aec198430 con 0x7f2aec1007b0 2024-08-16T19:09:33.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.090+0000 7f2aeb7fe640 1 -- 172.21.15.62:0/3835994296 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2adc09cac0 con 0x7f2aec1007b0 2024-08-16T19:09:33.091 INFO:teuthology.orchestra.run.smithi062.stdout:133143986238 2024-08-16T19:09:33.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 msgr2=0x7f2ad40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 0x7f2ad40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009900 tx=0x7f2ae00058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 msgr2=0x7f2aec1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 0x7f2aec1970d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00eb00 tx=0x7f2adc00c2d0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 shutdown_connections 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aec1067b0 0x7f2aec19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2aec101150 0x7f2aec197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f2ad40a2c90 0x7f2ad40a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 --2- 172.21.15.62:0/3835994296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2aec1007b0 0x7f2aec1970d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 >> 172.21.15.62:0/3835994296 conn(0x7f2aec0fc4d0 msgr2=0x7f2aec0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 shutdown_connections 2024-08-16T19:09:33.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.092+0000 7f2af4348640 1 -- 172.21.15.62:0/3835994296 wait complete. 2024-08-16T19:09:33.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593779 got 184683593781 for osd.6 2024-08-16T19:09:33.439 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:33.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:33 smithi062 ceph-mon[32993]: pgmap v307: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-08-16T19:09:33.722 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:33 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3835994296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 -- 172.21.15.62:0/597023471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41052e0 msgr2=0x7f29e410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/597023471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41052e0 0x7f29e410ce60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ca0 tx=0x7f29dc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 -- 172.21.15.62:0/597023471 shutdown_connections 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/597023471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41052e0 0x7f29e410ce60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/597023471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 0x7f29e4104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/597023471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41001b0 0x7f29e4100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 -- 172.21.15.62:0/597023471 >> 172.21.15.62:0/597023471 conn(0x7f29e4076f40 msgr2=0x7f29e40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:33.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 -- 172.21.15.62:0/597023471 shutdown_connections 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 -- 172.21.15.62:0/597023471 wait complete. 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.806+0000 7f29e9ff2640 1 Processor -- start 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 -- start start 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 0x7f29e40fc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41052e0 0x7f29e40fce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4110bc0 con 0x7f29e41001b0 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4110a40 con 0x7f29e4100b60 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e9ff2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4110d40 con 0x7f29e41052e0 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50642/0 (socket says 172.21.15.62:50642) 2024-08-16T19:09:33.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 -- 172.21.15.62:0/1399604073 learned_addr learned my addr 172.21.15.62:0/1399604073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e2ffd640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 0x7f29e40fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e3fff640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41052e0 0x7f29e40fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 -- 172.21.15.62:0/1399604073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41052e0 msgr2=0x7f29e40fce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41052e0 0x7f29e40fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 -- 172.21.15.62:0/1399604073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 msgr2=0x7f29e40fc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 0x7f29e40fc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.807+0000 7f29e37fe640 1 -- 172.21.15.62:0/1399604073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41a6650 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e37fe640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017030 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d4004020 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e41a6940 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d40047a0 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.808+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e41a6e80 con 0x7f29e41001b0 2024-08-16T19:09:33.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.809+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4100590 con 0x7f29e41001b0 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.810+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f29d40041d0 con 0x7f29e41001b0 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.811+0000 7f29e0ff9640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 0x7f29d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.811+0000 7f29e2ffd640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 0x7f29d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.811+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f29d40cef10 con 0x7f29e41001b0 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.812+0000 7f29e2ffd640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 0x7f29d00a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29e40fde50 tx=0x7f29cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.815+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f29d409b9b0 con 0x7f29e41001b0 2024-08-16T19:09:33.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:33 smithi155 ceph-mon[37413]: pgmap v307: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-08-16T19:09:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:33 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3835994296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-16T19:09:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:33 smithi161 ceph-mon[37576]: pgmap v307: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-08-16T19:09:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:33 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3835994296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-16T19:09:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.994+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f29e410a4d0 con 0x7f29e41001b0 2024-08-16T19:09:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.995+0000 7f29e0ff9640 1 -- 172.21.15.62:0/1399604073 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f29d4095e20 con 0x7f29e41001b0 2024-08-16T19:09:33.996 INFO:teuthology.orchestra.run.smithi062.stdout:326417514527 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 msgr2=0x7f29d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 0x7f29d00a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29e40fde50 tx=0x7f29cc005f90 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 msgr2=0x7f29e40fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f29d400d950 tx=0x7f29d400de10 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 shutdown_connections 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29e41052e0 0x7f29e40fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29d00a2c10 0x7f29d00a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e4100b60 0x7f29e40fc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 --2- 172.21.15.62:0/1399604073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29e41001b0 0x7f29e40fc3c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 >> 172.21.15.62:0/1399604073 conn(0x7f29e4076f40 msgr2=0x7f29e40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 shutdown_connections 2024-08-16T19:09:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:33.997+0000 7f29e9ff2640 1 -- 172.21.15.62:0/1399604073 wait complete. 2024-08-16T19:09:34.277 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986235 got 133143986238 for osd.4 2024-08-16T19:09:34.277 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:34.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.441+0000 7f82c8e5d640 1 -- 172.21.15.62:0/3786278866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4103410 msgr2=0x7f82c41078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.441+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/3786278866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4103410 0x7f82c41078c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009930 tx=0x7f82b802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 -- 172.21.15.62:0/3786278866 shutdown_connections 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/3786278866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4107fd0 0x7f82c410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/3786278866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4103410 0x7f82c41078c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/3786278866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4102a60 0x7f82c4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 -- 172.21.15.62:0/3786278866 >> 172.21.15.62:0/3786278866 conn(0x7f82c40fc500 msgr2=0x7f82c40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 -- 172.21.15.62:0/3786278866 shutdown_connections 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 -- 172.21.15.62:0/3786278866 wait complete. 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 Processor -- start 2024-08-16T19:09:34.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.442+0000 7f82c8e5d640 1 -- start start 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 0x7f82c419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 0x7f82c419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111de0 con 0x7f82c4102a60 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111c60 con 0x7f82c4103410 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c8e5d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111f60 con 0x7f82c4107fd0 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50668/0 (socket says 172.21.15.62:50668) 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 -- 172.21.15.62:0/423825392 learned_addr learned my addr 172.21.15.62:0/423825392 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c2d9d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 0x7f82c419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c1d9b640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 0x7f82c419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 -- 172.21.15.62:0/423825392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 msgr2=0x7f82c419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 0x7f82c419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 -- 172.21.15.62:0/423825392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 msgr2=0x7f82c419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 0x7f82c419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 -- 172.21.15.62:0/423825392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c419f430 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c2d9d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 0x7f82c419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c1d9b640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 0x7f82c419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82c259c640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c920 tx=0x7f82b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.443+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4007c70 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.444+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c419f720 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.444+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b4007e10 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.444+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4018890 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.444+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a7000 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.445+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c4102e40 con 0x7f82c4102a60 2024-08-16T19:09:34.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.446+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f82b4018a70 con 0x7f82c4102a60 2024-08-16T19:09:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.447+0000 7f82b37fe640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 0x7f82ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.447+0000 7f82c1d9b640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 0x7f82ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.447+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f82b40cf730 con 0x7f82c4102a60 2024-08-16T19:09:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.447+0000 7f82c1d9b640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 0x7f82ac0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82b802a940 tx=0x7f82b8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:34.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.451+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f82b4014030 con 0x7f82c4102a60 2024-08-16T19:09:34.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.635+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f82c419b7e0 con 0x7f82c4102a60 2024-08-16T19:09:34.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.636+0000 7f82b37fe640 1 -- 172.21.15.62:0/423825392 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82b400de20 con 0x7f82c4102a60 2024-08-16T19:09:34.636 INFO:teuthology.orchestra.run.smithi062.stdout:55834574921 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 msgr2=0x7f82ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 0x7f82ac0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82b802a940 tx=0x7f82b8005fd0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 msgr2=0x7f82c419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c920 tx=0x7f82b400cde0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 shutdown_connections 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82c4107fd0 0x7f82c419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f82ac0a2c10 0x7f82ac0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82c4103410 0x7f82c419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 --2- 172.21.15.62:0/423825392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c4102a60 0x7f82c419a480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:34.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 >> 172.21.15.62:0/423825392 conn(0x7f82c40fc500 msgr2=0x7f82c41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:34.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 shutdown_connections 2024-08-16T19:09:34.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:34.638+0000 7f82c8e5d640 1 -- 172.21.15.62:0/423825392 wait complete. 2024-08-16T19:09:34.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:34 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1399604073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-16T19:09:34.830 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:34 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1399604073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-16T19:09:34.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201321 got 236223201323 for osd.8 2024-08-16T19:09:34.858 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:34.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:34 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1399604073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-16T19:09:35.462 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:35.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514525 got 326417514527 for osd.11 2024-08-16T19:09:35.463 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:35 smithi062 ceph-mon[32993]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-16T19:09:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:35 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/423825392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-16T19:09:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:35 smithi155 ceph-mon[37413]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-16T19:09:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:35 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/423825392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-16T19:09:35.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822713 for osd.5 2024-08-16T19:09:35.880 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:35 smithi161 ceph-mon[37576]: pgmap v308: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-16T19:09:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:35 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/423825392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-16T19:09:36.071 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:36.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574921 for osd.1 2024-08-16T19:09:36.520 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 -- 172.21.15.62:0/3985234158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074102a60 msgr2=0x7f9074102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 --2- 172.21.15.62:0/3985234158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074102a60 0x7f9074102e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9064009930 tx=0x7f906402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 -- 172.21.15.62:0/3985234158 shutdown_connections 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 --2- 172.21.15.62:0/3985234158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074107fd0 0x7f907410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 --2- 172.21.15.62:0/3985234158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 0x7f90741078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 --2- 172.21.15.62:0/3985234158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074102a60 0x7f9074102e40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 -- 172.21.15.62:0/3985234158 >> 172.21.15.62:0/3985234158 conn(0x7f90740fc500 msgr2=0x7f90740fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 -- 172.21.15.62:0/3985234158 shutdown_connections 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 -- 172.21.15.62:0/3985234158 wait complete. 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.998+0000 7f9079c66640 1 Processor -- start 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 -- start start 2024-08-16T19:09:37.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074102a60 0x7f90741982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 0x7f90741987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9074111dd0 con 0x7f9074107fd0 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9074111c50 con 0x7f9074102a60 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9079c66640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9074111f50 con 0x7f9074103410 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50688/0 (socket says 172.21.15.62:50688) 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 -- 172.21.15.62:0/3627109986 learned_addr learned my addr 172.21.15.62:0/3627109986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f90737fe640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074102a60 0x7f90741982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9072ffd640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 0x7f90741987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 -- 172.21.15.62:0/3627109986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 msgr2=0x7f90741987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 0x7f90741987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 -- 172.21.15.62:0/3627109986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074102a60 msgr2=0x7f90741982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:37.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074102a60 0x7f90741982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:36.999+0000 7f9073fff640 1 -- 172.21.15.62:0/3627109986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907419d260 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9073fff640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c017070 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907419d550 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c003e50 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c010460 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.000+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a4e30 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.001+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907406a580 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.002+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f906c003970 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.003+0000 7f9070ff9640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 0x7f905c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.003+0000 7f90737fe640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 0x7f905c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.003+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f906c0cf080 con 0x7f9074107fd0 2024-08-16T19:09:37.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.004+0000 7f90737fe640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 0x7f905c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f90640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:37.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.006+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f906c09bb20 con 0x7f9074107fd0 2024-08-16T19:09:37.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.186+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9074109fe0 con 0x7f9074107fd0 2024-08-16T19:09:37.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.186+0000 7f9070ff9640 1 -- 172.21.15.62:0/3627109986 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f906c09b420 con 0x7f9074107fd0 2024-08-16T19:09:37.188 INFO:teuthology.orchestra.run.smithi062.stdout:34359738447 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 msgr2=0x7f905c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 0x7f905c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f90640058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 msgr2=0x7f907419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c00def0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 shutdown_connections 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9074107fd0 0x7f907419cb80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9074103410 0x7f90741987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f905c0a2c10 0x7f905c0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 --2- 172.21.15.62:0/3627109986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9074102a60 0x7f90741982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.188+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 >> 172.21.15.62:0/3627109986 conn(0x7f90740fc500 msgr2=0x7f9074105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.189+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 shutdown_connections 2024-08-16T19:09:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:37.189+0000 7f9079c66640 1 -- 172.21.15.62:0/3627109986 wait complete. 2024-08-16T19:09:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:37 smithi155 ceph-mon[37413]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:37 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3627109986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-16T19:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:37 smithi062 ceph-mon[32993]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:37.863 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:37 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3627109986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-16T19:09:37.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:37 smithi161 ceph-mon[37576]: pgmap v309: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:37 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3627109986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-16T19:09:37.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738447 for osd.0 2024-08-16T19:09:37.891 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 -- 172.21.15.62:0/830495439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0102a70 msgr2=0x7fd9e0102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 --2- 172.21.15.62:0/830495439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0102a70 0x7fd9e0102e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009900 tx=0x7fd9d402a480 comp rx=0 tx=0).stop 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 -- 172.21.15.62:0/830495439 shutdown_connections 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 --2- 172.21.15.62:0/830495439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 0x7fd9e010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 --2- 172.21.15.62:0/830495439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0103390 0x7fd9e010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 --2- 172.21.15.62:0/830495439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0102a70 0x7fd9e0102e50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 -- 172.21.15.62:0/830495439 >> 172.21.15.62:0/830495439 conn(0x7fd9e00fc4d0 msgr2=0x7fd9e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.011+0000 7fd9e5393640 1 -- 172.21.15.62:0/830495439 shutdown_connections 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 -- 172.21.15.62:0/830495439 wait complete. 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 Processor -- start 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 -- start start 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0102a70 0x7fd9e0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 0x7fd9e019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0111ca0 con 0x7fd9e0103390 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0111b20 con 0x7fd9e010bea0 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.012+0000 7fd9e5393640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e0111e20 con 0x7fd9e0102a70 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:38.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50702/0 (socket says 172.21.15.62:50702) 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 -- 172.21.15.62:0/559839068 learned_addr learned my addr 172.21.15.62:0/559839068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9deffd640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0102a70 0x7fd9e0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9df7fe640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 0x7fd9e019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 -- 172.21.15.62:0/559839068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0102a70 msgr2=0x7fd9e0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0102a70 0x7fd9e0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 -- 172.21.15.62:0/559839068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 msgr2=0x7fd9e019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 0x7fd9e019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 -- 172.21.15.62:0/559839068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e019d0e0 con 0x7fd9e0103390 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9de7fc640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c940 tx=0x7fd9d000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004220 con 0x7fd9e0103390 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d00043c0 con 0x7fd9e0103390 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e019d3d0 con 0x7fd9e0103390 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.013+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004a00 con 0x7fd9e0103390 2024-08-16T19:09:38.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.014+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e01a4cb0 con 0x7fd9e0103390 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.015+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e0102e50 con 0x7fd9e0103390 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.016+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd9d00079b0 con 0x7fd9e0103390 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.017+0000 7fd9c7fff640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 0x7fd9c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.017+0000 7fd9deffd640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 0x7fd9c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.017+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fd9d00cf480 con 0x7fd9e0103390 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.017+0000 7fd9deffd640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 0x7fd9c80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009230 tx=0x7fd9d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:38.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.021+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd9d0014030 con 0x7fd9e0103390 2024-08-16T19:09:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.198+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd9e0106980 con 0x7fd9e0103390 2024-08-16T19:09:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.198+0000 7fd9c7fff640 1 -- 172.21.15.62:0/559839068 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd9d009b8d0 con 0x7fd9e0103390 2024-08-16T19:09:38.200 INFO:teuthology.orchestra.run.smithi062.stdout:292057776163 2024-08-16T19:09:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.200+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 msgr2=0x7fd9c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.200+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 0x7fd9c80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009230 tx=0x7fd9d40058d0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.200+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 msgr2=0x7fd9e0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.200+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c940 tx=0x7fd9d000ce00 comp rx=0 tx=0).stop 2024-08-16T19:09:38.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 shutdown_connections 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fd9c80a2c10 0x7fd9c80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9e010bea0 0x7fd9e019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9e0103390 0x7fd9e0198670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 --2- 172.21.15.62:0/559839068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9e0102a70 0x7fd9e0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 >> 172.21.15.62:0/559839068 conn(0x7fd9e00fc4d0 msgr2=0x7fd9e00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 shutdown_connections 2024-08-16T19:09:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:38.201+0000 7fd9e5393640 1 -- 172.21.15.62:0/559839068 wait complete. 2024-08-16T19:09:38.794 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:38 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/559839068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-16T19:09:38.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776163 for osd.10 2024-08-16T19:09:38.824 DEBUG:teuthology.parallel:result is None 2024-08-16T19:09:38.824 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-16T19:09:38.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:09:38.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:38 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/559839068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-16T19:09:38.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:38 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/559839068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-16T19:09:39.091 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 -- 172.21.15.62:0/730451378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 msgr2=0x7f9e08100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:39.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/730451378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e08100c20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009930 tx=0x7f9df802a420 comp rx=0 tx=0).stop 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 -- 172.21.15.62:0/730451378 shutdown_connections 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/730451378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e0810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/730451378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e08100c20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/730451378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 0x7f9e08106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 -- 172.21.15.62:0/730451378 >> 172.21.15.62:0/730451378 conn(0x7f9e080fc500 msgr2=0x7f9e080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 -- 172.21.15.62:0/730451378 shutdown_connections 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.504+0000 7f9e0fec0640 1 -- 172.21.15.62:0/730451378 wait complete. 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 Processor -- start 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 -- start start 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e080fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 0x7f9e080fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:39.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08110bc0 con 0x7f9e081007c0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08110a40 con 0x7f9e08101160 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0fec0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08110d40 con 0x7f9e081067c0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50724/0 (socket says 172.21.15.62:50724) 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 -- 172.21.15.62:0/618183401 learned_addr learned my addr 172.21.15.62:0/618183401 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0e45d640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 0x7f9e080fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0d45b640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e080fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 -- 172.21.15.62:0/618183401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 msgr2=0x7f9e080fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 0x7f9e080fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 -- 172.21.15.62:0/618183401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 msgr2=0x7f9e080fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e080fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0dc5c640 1 -- 172.21.15.62:0/618183401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e081a6620 con 0x7f9e081007c0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.505+0000 7f9e0d45b640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e080fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9e0dc5c640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400d770 tx=0x7f9e0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:39.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04017070 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e04005500 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e081a6910 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e04004e00 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.506+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a6dd0 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.507+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08100c20 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.508+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e04004fa0 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.509+0000 7f9dfeffd640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 0x7f9df40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.509+0000 7f9e0d45b640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 0x7f9df40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.509+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f9e040cf340 con 0x7f9e081007c0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.510+0000 7f9e0d45b640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 0x7f9df40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:39.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.512+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e0409be80 con 0x7f9e081007c0 2024-08-16T19:09:39.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.688+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e08062360 con 0x7f9df40a2c10 2024-08-16T19:09:39.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.697+0000 7f9dfeffd640 1 -- 172.21.15.62:0/618183401 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340404 (secure 0 0 0) 0x7f9e08062360 con 0x7f9df40a2c10 2024-08-16T19:09:39.702 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:39.703 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:09:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 msgr2=0x7f9df40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 0x7f9df40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8005a20 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 msgr2=0x7f9e080fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400d770 tx=0x7f9e0400dc30 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 shutdown_connections 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e081067c0 0x7f9e080fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f9df40a2c10 0x7f9df40a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9e08101160 0x7f9e080fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 --2- 172.21.15.62:0/618183401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e081007c0 0x7f9e080fcb40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 >> 172.21.15.62:0/618183401 conn(0x7f9e080fc500 msgr2=0x7f9e0806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 shutdown_connections 2024-08-16T19:09:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:39.701+0000 7f9e0fec0640 1 -- 172.21.15.62:0/618183401 wait complete. 2024-08-16T19:09:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:39 smithi155 ceph-mon[37413]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:39 smithi161 ceph-mon[37576]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:39 smithi062 ceph-mon[32993]: pgmap v310: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:40.356 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-08-16T19:09:39.670532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660264,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":673,"num_read_kb":468,"num_write":432,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375920,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565904,"statfs":{"total":1151940427776,"available":1151555485696,"internally_reserved":0,"allocated":14569472,"data_stored":8052818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003444"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.487171+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:09:33.487171+0000","last_peered":"2024-08-16T19:09:33.487171+0000","last_clean":"2024-08-16T19:09:33.487171+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:09:33.487171+0000","last_undegraded":"2024-08-16T19:09:33.487171+0000","last_fullsized":"2024-08-16T19:09:33.487171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592994+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:08:59.592994+0000","last_peered":"2024-08-16T19:08:59.592994+0000","last_clean":"2024-08-16T19:08:59.592994+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:08:59.592994+0000","last_undegraded":"2024-08-16T19:08:59.592994+0000","last_fullsized":"2024-08-16T19:08:59.592994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456279+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:08:59.456279+0000","last_peered":"2024-08-16T19:08:59.456279+0000","last_clean":"2024-08-16T19:08:59.456279+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:08:59.456279+0000","last_undegraded":"2024-08-16T19:08:59.456279+0000","last_fullsized":"2024-08-16T19:08:59.456279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593258+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:08:59.593258+0000","last_peered":"2024-08-16T19:08:59.593258+0000","last_clean":"2024-08-16T19:08:59.593258+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:08:59.593258+0000","last_undegraded":"2024-08-16T19:08:59.593258+0000","last_fullsized":"2024-08-16T19:08:59.593258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458615+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:08:59.458615+0000","last_peered":"2024-08-16T19:08:59.458615+0000","last_clean":"2024-08-16T19:08:59.458615+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:08:59.458615+0000","last_undegraded":"2024-08-16T19:08:59.458615+0000","last_fullsized":"2024-08-16T19:08:59.458615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.594556+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:08:59.594556+0000","last_peered":"2024-08-16T19:08:59.594556+0000","last_clean":"2024-08-16T19:08:59.594556+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:08:59.594556+0000","last_undegraded":"2024-08-16T19:08:59.594556+0000","last_fullsized":"2024-08-16T19:08:59.594556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459473+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:08:59.459473+0000","last_peered":"2024-08-16T19:08:59.459473+0000","last_clean":"2024-08-16T19:08:59.459473+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:08:59.459473+0000","last_undegraded":"2024-08-16T19:08:59.459473+0000","last_fullsized":"2024-08-16T19:08:59.459473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545800+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:08:59.545800+0000","last_peered":"2024-08-16T19:08:59.545800+0000","last_clean":"2024-08-16T19:08:59.545800+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:08:59.545800+0000","last_undegraded":"2024-08-16T19:08:59.545800+0000","last_fullsized":"2024-08-16T19:08:59.545800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457100+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:08:59.457100+0000","last_peered":"2024-08-16T19:08:59.457100+0000","last_clean":"2024-08-16T19:08:59.457100+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:08:59.457100+0000","last_undegraded":"2024-08-16T19:08:59.457100+0000","last_fullsized":"2024-08-16T19:08:59.457100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459443+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:08:59.459443+0000","last_peered":"2024-08-16T19:08:59.459443+0000","last_clean":"2024-08-16T19:08:59.459443+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:08:59.459443+0000","last_undegraded":"2024-08-16T19:08:59.459443+0000","last_fullsized":"2024-08-16T19:08:59.459443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.547859+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:08:59.547859+0000","last_peered":"2024-08-16T19:08:59.547859+0000","last_clean":"2024-08-16T19:08:59.547859+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:08:59.547859+0000","last_undegraded":"2024-08-16T19:08:59.547859+0000","last_fullsized":"2024-08-16T19:08:59.547859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457189+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:08:59.457189+0000","last_peered":"2024-08-16T19:08:59.457189+0000","last_clean":"2024-08-16T19:08:59.457189+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:08:59.457189+0000","last_undegraded":"2024-08-16T19:08:59.457189+0000","last_fullsized":"2024-08-16T19:08:59.457189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454902+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:08:59.454902+0000","last_peered":"2024-08-16T19:08:59.454902+0000","last_clean":"2024-08-16T19:08:59.454902+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:08:59.454902+0000","last_undegraded":"2024-08-16T19:08:59.454902+0000","last_fullsized":"2024-08-16T19:08:59.454902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458666+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:08:59.458666+0000","last_peered":"2024-08-16T19:08:59.458666+0000","last_clean":"2024-08-16T19:08:59.458666+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:08:59.458666+0000","last_undegraded":"2024-08-16T19:08:59.458666+0000","last_fullsized":"2024-08-16T19:08:59.458666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457164+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:08:59.457164+0000","last_peered":"2024-08-16T19:08:59.457164+0000","last_clean":"2024-08-16T19:08:59.457164+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:08:59.457164+0000","last_undegraded":"2024-08-16T19:08:59.457164+0000","last_fullsized":"2024-08-16T19:08:59.457164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592793+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:08:59.592793+0000","last_peered":"2024-08-16T19:08:59.592793+0000","last_clean":"2024-08-16T19:08:59.592793+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:08:59.592793+0000","last_undegraded":"2024-08-16T19:08:59.592793+0000","last_fullsized":"2024-08-16T19:08:59.592793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456614+0000","last_change":"2024-08-16T19:08:02.462372+0000","last_active":"2024-08-16T19:08:59.456614+0000","last_peered":"2024-08-16T19:08:59.456614+0000","last_clean":"2024-08-16T19:08:59.456614+0000","last_became_active":"2024-08-16T19:08:02.461973+0000","last_became_peered":"2024-08-16T19:08:02.461973+0000","last_unstale":"2024-08-16T19:08:59.456614+0000","last_undegraded":"2024-08-16T19:08:59.456614+0000","last_fullsized":"2024-08-16T19:08:59.456614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:24:04.191300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457135+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:08:59.457135+0000","last_peered":"2024-08-16T19:08:59.457135+0000","last_clean":"2024-08-16T19:08:59.457135+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:08:59.457135+0000","last_undegraded":"2024-08-16T19:08:59.457135+0000","last_fullsized":"2024-08-16T19:08:59.457135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462630+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:08:59.462630+0000","last_peered":"2024-08-16T19:08:59.462630+0000","last_clean":"2024-08-16T19:08:59.462630+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:08:59.462630+0000","last_undegraded":"2024-08-16T19:08:59.462630+0000","last_fullsized":"2024-08-16T19:08:59.462630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593234+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:08:59.593234+0000","last_peered":"2024-08-16T19:08:59.593234+0000","last_clean":"2024-08-16T19:08:59.593234+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:08:59.593234+0000","last_undegraded":"2024-08-16T19:08:59.593234+0000","last_fullsized":"2024-08-16T19:08:59.593234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457650+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:08:59.457650+0000","last_peered":"2024-08-16T19:08:59.457650+0000","last_clean":"2024-08-16T19:08:59.457650+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:08:59.457650+0000","last_undegraded":"2024-08-16T19:08:59.457650+0000","last_fullsized":"2024-08-16T19:08:59.457650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458319+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:08:59.458319+0000","last_peered":"2024-08-16T19:08:59.458319+0000","last_clean":"2024-08-16T19:08:59.458319+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:08:59.458319+0000","last_undegraded":"2024-08-16T19:08:59.458319+0000","last_fullsized":"2024-08-16T19:08:59.458319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545351+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:08:59.545351+0000","last_peered":"2024-08-16T19:08:59.545351+0000","last_clean":"2024-08-16T19:08:59.545351+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:08:59.545351+0000","last_undegraded":"2024-08-16T19:08:59.545351+0000","last_fullsized":"2024-08-16T19:08:59.545351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.464030+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:08:59.464030+0000","last_peered":"2024-08-16T19:08:59.464030+0000","last_clean":"2024-08-16T19:08:59.464030+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:08:59.464030+0000","last_undegraded":"2024-08-16T19:08:59.464030+0000","last_fullsized":"2024-08-16T19:08:59.464030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458222+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:08:59.458222+0000","last_peered":"2024-08-16T19:08:59.458222+0000","last_clean":"2024-08-16T19:08:59.458222+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:08:59.458222+0000","last_undegraded":"2024-08-16T19:08:59.458222+0000","last_fullsized":"2024-08-16T19:08:59.458222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458400+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:08:59.458400+0000","last_peered":"2024-08-16T19:08:59.458400+0000","last_clean":"2024-08-16T19:08:59.458400+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:08:59.458400+0000","last_undegraded":"2024-08-16T19:08:59.458400+0000","last_fullsized":"2024-08-16T19:08:59.458400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456408+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:08:59.456408+0000","last_peered":"2024-08-16T19:08:59.456408+0000","last_clean":"2024-08-16T19:08:59.456408+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:08:59.456408+0000","last_undegraded":"2024-08-16T19:08:59.456408+0000","last_fullsized":"2024-08-16T19:08:59.456408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466015+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:08:59.466015+0000","last_peered":"2024-08-16T19:08:59.466015+0000","last_clean":"2024-08-16T19:08:59.466015+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:08:59.466015+0000","last_undegraded":"2024-08-16T19:08:59.466015+0000","last_fullsized":"2024-08-16T19:08:59.466015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:00.591976+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:09:00.591976+0000","last_peered":"2024-08-16T19:09:00.591976+0000","last_clean":"2024-08-16T19:09:00.591976+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:09:00.591976+0000","last_undegraded":"2024-08-16T19:09:00.591976+0000","last_fullsized":"2024-08-16T19:09:00.591976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593074+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:08:59.593074+0000","last_peered":"2024-08-16T19:08:59.593074+0000","last_clean":"2024-08-16T19:08:59.593074+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:08:59.593074+0000","last_undegraded":"2024-08-16T19:08:59.593074+0000","last_fullsized":"2024-08-16T19:08:59.593074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455577+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:08:59.455577+0000","last_peered":"2024-08-16T19:08:59.455577+0000","last_clean":"2024-08-16T19:08:59.455577+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:08:59.455577+0000","last_undegraded":"2024-08-16T19:08:59.455577+0000","last_fullsized":"2024-08-16T19:08:59.455577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459679+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:08:59.459679+0000","last_peered":"2024-08-16T19:08:59.459679+0000","last_clean":"2024-08-16T19:08:59.459679+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:08:59.459679+0000","last_undegraded":"2024-08-16T19:08:59.459679+0000","last_fullsized":"2024-08-16T19:08:59.459679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462351+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:08:59.462351+0000","last_peered":"2024-08-16T19:08:59.462351+0000","last_clean":"2024-08-16T19:08:59.462351+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:08:59.462351+0000","last_undegraded":"2024-08-16T19:08:59.462351+0000","last_fullsized":"2024-08-16T19:08:59.462351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458157+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:08:59.458157+0000","last_peered":"2024-08-16T19:08:59.458157+0000","last_clean":"2024-08-16T19:08:59.458157+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:08:59.458157+0000","last_undegraded":"2024-08-16T19:08:59.458157+0000","last_fullsized":"2024-08-16T19:08:59.458157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456378+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:08:59.456378+0000","last_peered":"2024-08-16T19:08:59.456378+0000","last_clean":"2024-08-16T19:08:59.456378+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:08:59.456378+0000","last_undegraded":"2024-08-16T19:08:59.456378+0000","last_fullsized":"2024-08-16T19:08:59.456378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458335+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:08:59.458335+0000","last_peered":"2024-08-16T19:08:59.458335+0000","last_clean":"2024-08-16T19:08:59.458335+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:08:59.458335+0000","last_undegraded":"2024-08-16T19:08:59.458335+0000","last_fullsized":"2024-08-16T19:08:59.458335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457766+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:08:59.457766+0000","last_peered":"2024-08-16T19:08:59.457766+0000","last_clean":"2024-08-16T19:08:59.457766+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:08:59.457766+0000","last_undegraded":"2024-08-16T19:08:59.457766+0000","last_fullsized":"2024-08-16T19:08:59.457766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.487635+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:09:33.487635+0000","last_peered":"2024-08-16T19:09:33.487635+0000","last_clean":"2024-08-16T19:09:33.487635+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:09:33.487635+0000","last_undegraded":"2024-08-16T19:09:33.487635+0000","last_fullsized":"2024-08-16T19:09:33.487635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455413+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:08:59.455413+0000","last_peered":"2024-08-16T19:08:59.455413+0000","last_clean":"2024-08-16T19:08:59.455413+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:08:59.455413+0000","last_undegraded":"2024-08-16T19:08:59.455413+0000","last_fullsized":"2024-08-16T19:08:59.455413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454239+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:08:59.454239+0000","last_peered":"2024-08-16T19:08:59.454239+0000","last_clean":"2024-08-16T19:08:59.454239+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:08:59.454239+0000","last_undegraded":"2024-08-16T19:08:59.454239+0000","last_fullsized":"2024-08-16T19:08:59.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.481408+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:09:36.481408+0000","last_peered":"2024-08-16T19:09:36.481408+0000","last_clean":"2024-08-16T19:09:36.481408+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:09:36.481408+0000","last_undegraded":"2024-08-16T19:09:36.481408+0000","last_fullsized":"2024-08-16T19:09:36.481408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455963+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:08:59.455963+0000","last_peered":"2024-08-16T19:08:59.455963+0000","last_clean":"2024-08-16T19:08:59.455963+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:08:59.455963+0000","last_undegraded":"2024-08-16T19:08:59.455963+0000","last_fullsized":"2024-08-16T19:08:59.455963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455170+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:08:59.455170+0000","last_peered":"2024-08-16T19:08:59.455170+0000","last_clean":"2024-08-16T19:08:59.455170+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:08:59.455170+0000","last_undegraded":"2024-08-16T19:08:59.455170+0000","last_fullsized":"2024-08-16T19:08:59.455170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454875+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:08:59.454875+0000","last_peered":"2024-08-16T19:08:59.454875+0000","last_clean":"2024-08-16T19:08:59.454875+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:08:59.454875+0000","last_undegraded":"2024-08-16T19:08:59.454875+0000","last_fullsized":"2024-08-16T19:08:59.454875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454959+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:08:59.454959+0000","last_peered":"2024-08-16T19:08:59.454959+0000","last_clean":"2024-08-16T19:08:59.454959+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:08:59.454959+0000","last_undegraded":"2024-08-16T19:08:59.454959+0000","last_fullsized":"2024-08-16T19:08:59.454959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592928+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:08:59.592928+0000","last_peered":"2024-08-16T19:08:59.592928+0000","last_clean":"2024-08-16T19:08:59.592928+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:08:59.592928+0000","last_undegraded":"2024-08-16T19:08:59.592928+0000","last_fullsized":"2024-08-16T19:08:59.592928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458401+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:08:59.458401+0000","last_peered":"2024-08-16T19:08:59.458401+0000","last_clean":"2024-08-16T19:08:59.458401+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:08:59.458401+0000","last_undegraded":"2024-08-16T19:08:59.458401+0000","last_fullsized":"2024-08-16T19:08:59.458401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455187+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:08:59.455187+0000","last_peered":"2024-08-16T19:08:59.455187+0000","last_clean":"2024-08-16T19:08:59.455187+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:08:59.455187+0000","last_undegraded":"2024-08-16T19:08:59.455187+0000","last_fullsized":"2024-08-16T19:08:59.455187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455400+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:08:59.455400+0000","last_peered":"2024-08-16T19:08:59.455400+0000","last_clean":"2024-08-16T19:08:59.455400+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:08:59.455400+0000","last_undegraded":"2024-08-16T19:08:59.455400+0000","last_fullsized":"2024-08-16T19:08:59.455400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455750+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:08:59.455750+0000","last_peered":"2024-08-16T19:08:59.455750+0000","last_clean":"2024-08-16T19:08:59.455750+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:08:59.455750+0000","last_undegraded":"2024-08-16T19:08:59.455750+0000","last_fullsized":"2024-08-16T19:08:59.455750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:35.494168+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:09:35.494168+0000","last_peered":"2024-08-16T19:09:35.494168+0000","last_clean":"2024-08-16T19:09:35.494168+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:09:35.494168+0000","last_undegraded":"2024-08-16T19:09:35.494168+0000","last_fullsized":"2024-08-16T19:09:35.494168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545422+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:08:59.545422+0000","last_peered":"2024-08-16T19:08:59.545422+0000","last_clean":"2024-08-16T19:08:59.545422+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:08:59.545422+0000","last_undegraded":"2024-08-16T19:08:59.545422+0000","last_fullsized":"2024-08-16T19:08:59.545422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455331+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:08:59.455331+0000","last_peered":"2024-08-16T19:08:59.455331+0000","last_clean":"2024-08-16T19:08:59.455331+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:08:59.455331+0000","last_undegraded":"2024-08-16T19:08:59.455331+0000","last_fullsized":"2024-08-16T19:08:59.455331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454825+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:08:59.454825+0000","last_peered":"2024-08-16T19:08:59.454825+0000","last_clean":"2024-08-16T19:08:59.454825+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:08:59.454825+0000","last_undegraded":"2024-08-16T19:08:59.454825+0000","last_fullsized":"2024-08-16T19:08:59.454825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457861+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:08:59.457861+0000","last_peered":"2024-08-16T19:08:59.457861+0000","last_clean":"2024-08-16T19:08:59.457861+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:08:59.457861+0000","last_undegraded":"2024-08-16T19:08:59.457861+0000","last_fullsized":"2024-08-16T19:08:59.457861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455492+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:08:59.455492+0000","last_peered":"2024-08-16T19:08:59.455492+0000","last_clean":"2024-08-16T19:08:59.455492+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:08:59.455492+0000","last_undegraded":"2024-08-16T19:08:59.455492+0000","last_fullsized":"2024-08-16T19:08:59.455492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455979+0000","last_change":"2024-08-16T19:08:02.457345+0000","last_active":"2024-08-16T19:08:59.455979+0000","last_peered":"2024-08-16T19:08:59.455979+0000","last_clean":"2024-08-16T19:08:59.455979+0000","last_became_active":"2024-08-16T19:08:02.456961+0000","last_became_peered":"2024-08-16T19:08:02.456961+0000","last_unstale":"2024-08-16T19:08:59.455979+0000","last_undegraded":"2024-08-16T19:08:59.455979+0000","last_fullsized":"2024-08-16T19:08:59.455979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:24.143886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454067+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:08:59.454067+0000","last_peered":"2024-08-16T19:08:59.454067+0000","last_clean":"2024-08-16T19:08:59.454067+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:08:59.454067+0000","last_undegraded":"2024-08-16T19:08:59.454067+0000","last_fullsized":"2024-08-16T19:08:59.454067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462353+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:08:59.462353+0000","last_peered":"2024-08-16T19:08:59.462353+0000","last_clean":"2024-08-16T19:08:59.462353+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:08:59.462353+0000","last_undegraded":"2024-08-16T19:08:59.462353+0000","last_fullsized":"2024-08-16T19:08:59.462353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459864+0000","last_change":"2024-08-16T19:08:02.457999+0000","last_active":"2024-08-16T19:08:59.459864+0000","last_peered":"2024-08-16T19:08:59.459864+0000","last_clean":"2024-08-16T19:08:59.459864+0000","last_became_active":"2024-08-16T19:08:02.456740+0000","last_became_peered":"2024-08-16T19:08:02.456740+0000","last_unstale":"2024-08-16T19:08:59.459864+0000","last_undegraded":"2024-08-16T19:08:59.459864+0000","last_fullsized":"2024-08-16T19:08:59.459864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:20:22.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454151+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:08:59.454151+0000","last_peered":"2024-08-16T19:08:59.454151+0000","last_clean":"2024-08-16T19:08:59.454151+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:08:59.454151+0000","last_undegraded":"2024-08-16T19:08:59.454151+0000","last_fullsized":"2024-08-16T19:08:59.454151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454277+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:08:59.454277+0000","last_peered":"2024-08-16T19:08:59.454277+0000","last_clean":"2024-08-16T19:08:59.454277+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:08:59.454277+0000","last_undegraded":"2024-08-16T19:08:59.454277+0000","last_fullsized":"2024-08-16T19:08:59.454277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458557+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:08:59.458557+0000","last_peered":"2024-08-16T19:08:59.458557+0000","last_clean":"2024-08-16T19:08:59.458557+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:08:59.458557+0000","last_undegraded":"2024-08-16T19:08:59.458557+0000","last_fullsized":"2024-08-16T19:08:59.458557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455944+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:08:59.455944+0000","last_peered":"2024-08-16T19:08:59.455944+0000","last_clean":"2024-08-16T19:08:59.455944+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:08:59.455944+0000","last_undegraded":"2024-08-16T19:08:59.455944+0000","last_fullsized":"2024-08-16T19:08:59.455944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454634+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:08:59.454634+0000","last_peered":"2024-08-16T19:08:59.454634+0000","last_clean":"2024-08-16T19:08:59.454634+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:08:59.454634+0000","last_undegraded":"2024-08-16T19:08:59.454634+0000","last_fullsized":"2024-08-16T19:08:59.454634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.477295+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:09:33.477295+0000","last_peered":"2024-08-16T19:09:33.477295+0000","last_clean":"2024-08-16T19:09:33.477295+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:09:33.477295+0000","last_undegraded":"2024-08-16T19:09:33.477295+0000","last_fullsized":"2024-08-16T19:09:33.477295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595244+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:08:59.595244+0000","last_peered":"2024-08-16T19:08:59.595244+0000","last_clean":"2024-08-16T19:08:59.595244+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:08:59.595244+0000","last_undegraded":"2024-08-16T19:08:59.595244+0000","last_fullsized":"2024-08-16T19:08:59.595244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455402+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:08:59.455402+0000","last_peered":"2024-08-16T19:08:59.455402+0000","last_clean":"2024-08-16T19:08:59.455402+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:08:59.455402+0000","last_undegraded":"2024-08-16T19:08:59.455402+0000","last_fullsized":"2024-08-16T19:08:59.455402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454397+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:08:59.454397+0000","last_peered":"2024-08-16T19:08:59.454397+0000","last_clean":"2024-08-16T19:08:59.454397+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:08:59.454397+0000","last_undegraded":"2024-08-16T19:08:59.454397+0000","last_fullsized":"2024-08-16T19:08:59.454397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462052+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:08:59.462052+0000","last_peered":"2024-08-16T19:08:59.462052+0000","last_clean":"2024-08-16T19:08:59.462052+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:08:59.462052+0000","last_undegraded":"2024-08-16T19:08:59.462052+0000","last_fullsized":"2024-08-16T19:08:59.462052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455923+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:08:59.455923+0000","last_peered":"2024-08-16T19:08:59.455923+0000","last_clean":"2024-08-16T19:08:59.455923+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:08:59.455923+0000","last_undegraded":"2024-08-16T19:08:59.455923+0000","last_fullsized":"2024-08-16T19:08:59.455923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595109+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:08:59.595109+0000","last_peered":"2024-08-16T19:08:59.595109+0000","last_clean":"2024-08-16T19:08:59.595109+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:08:59.595109+0000","last_undegraded":"2024-08-16T19:08:59.595109+0000","last_fullsized":"2024-08-16T19:08:59.595109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459318+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:08:59.459318+0000","last_peered":"2024-08-16T19:08:59.459318+0000","last_clean":"2024-08-16T19:08:59.459318+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:08:59.459318+0000","last_undegraded":"2024-08-16T19:08:59.459318+0000","last_fullsized":"2024-08-16T19:08:59.459318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":178,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.486587+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:09:36.486587+0000","last_peered":"2024-08-16T19:09:36.486587+0000","last_clean":"2024-08-16T19:09:36.486587+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:09:36.486587+0000","last_undegraded":"2024-08-16T19:09:36.486587+0000","last_fullsized":"2024-08-16T19:09:36.486587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62070,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":117,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456560+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:08:59.456560+0000","last_peered":"2024-08-16T19:08:59.456560+0000","last_clean":"2024-08-16T19:08:59.456560+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:08:59.456560+0000","last_undegraded":"2024-08-16T19:08:59.456560+0000","last_fullsized":"2024-08-16T19:08:59.456560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546106+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:08:59.546106+0000","last_peered":"2024-08-16T19:08:59.546106+0000","last_clean":"2024-08-16T19:08:59.546106+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:08:59.546106+0000","last_undegraded":"2024-08-16T19:08:59.546106+0000","last_fullsized":"2024-08-16T19:08:59.546106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454804+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:08:59.454804+0000","last_peered":"2024-08-16T19:08:59.454804+0000","last_clean":"2024-08-16T19:08:59.454804+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:08:59.454804+0000","last_undegraded":"2024-08-16T19:08:59.454804+0000","last_fullsized":"2024-08-16T19:08:59.454804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458095+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:08:59.458095+0000","last_peered":"2024-08-16T19:08:59.458095+0000","last_clean":"2024-08-16T19:08:59.458095+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:08:59.458095+0000","last_undegraded":"2024-08-16T19:08:59.458095+0000","last_fullsized":"2024-08-16T19:08:59.458095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.460897+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:08:59.460897+0000","last_peered":"2024-08-16T19:08:59.460897+0000","last_clean":"2024-08-16T19:08:59.460897+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:08:59.460897+0000","last_undegraded":"2024-08-16T19:08:59.460897+0000","last_fullsized":"2024-08-16T19:08:59.460897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455542+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:08:59.455542+0000","last_peered":"2024-08-16T19:08:59.455542+0000","last_clean":"2024-08-16T19:08:59.455542+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:08:59.455542+0000","last_undegraded":"2024-08-16T19:08:59.455542+0000","last_fullsized":"2024-08-16T19:08:59.455542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456325+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:08:59.456325+0000","last_peered":"2024-08-16T19:08:59.456325+0000","last_clean":"2024-08-16T19:08:59.456325+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:08:59.456325+0000","last_undegraded":"2024-08-16T19:08:59.456325+0000","last_fullsized":"2024-08-16T19:08:59.456325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545951+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:08:59.545951+0000","last_peered":"2024-08-16T19:08:59.545951+0000","last_clean":"2024-08-16T19:08:59.545951+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:08:59.545951+0000","last_undegraded":"2024-08-16T19:08:59.545951+0000","last_fullsized":"2024-08-16T19:08:59.545951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.481558+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:09:36.481558+0000","last_peered":"2024-08-16T19:09:36.481558+0000","last_clean":"2024-08-16T19:09:36.481558+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:09:36.481558+0000","last_undegraded":"2024-08-16T19:09:36.481558+0000","last_fullsized":"2024-08-16T19:09:36.481558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592679+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:08:59.592679+0000","last_peered":"2024-08-16T19:08:59.592679+0000","last_clean":"2024-08-16T19:08:59.592679+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:08:59.592679+0000","last_undegraded":"2024-08-16T19:08:59.592679+0000","last_fullsized":"2024-08-16T19:08:59.592679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457785+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:08:59.457785+0000","last_peered":"2024-08-16T19:08:59.457785+0000","last_clean":"2024-08-16T19:08:59.457785+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:08:59.457785+0000","last_undegraded":"2024-08-16T19:08:59.457785+0000","last_fullsized":"2024-08-16T19:08:59.457785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456321+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:08:59.456321+0000","last_peered":"2024-08-16T19:08:59.456321+0000","last_clean":"2024-08-16T19:08:59.456321+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:08:59.456321+0000","last_undegraded":"2024-08-16T19:08:59.456321+0000","last_fullsized":"2024-08-16T19:08:59.456321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454450+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:08:59.454450+0000","last_peered":"2024-08-16T19:08:59.454450+0000","last_clean":"2024-08-16T19:08:59.454450+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:08:59.454450+0000","last_undegraded":"2024-08-16T19:08:59.454450+0000","last_fullsized":"2024-08-16T19:08:59.454450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462131+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:08:59.462131+0000","last_peered":"2024-08-16T19:08:59.462131+0000","last_clean":"2024-08-16T19:08:59.462131+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:08:59.462131+0000","last_undegraded":"2024-08-16T19:08:59.462131+0000","last_fullsized":"2024-08-16T19:08:59.462131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593351+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:08:59.593351+0000","last_peered":"2024-08-16T19:08:59.593351+0000","last_clean":"2024-08-16T19:08:59.593351+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:08:59.593351+0000","last_undegraded":"2024-08-16T19:08:59.593351+0000","last_fullsized":"2024-08-16T19:08:59.593351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455179+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:08:59.455179+0000","last_peered":"2024-08-16T19:08:59.455179+0000","last_clean":"2024-08-16T19:08:59.455179+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:08:59.455179+0000","last_undegraded":"2024-08-16T19:08:59.455179+0000","last_fullsized":"2024-08-16T19:08:59.455179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456946+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:08:59.456946+0000","last_peered":"2024-08-16T19:08:59.456946+0000","last_clean":"2024-08-16T19:08:59.456946+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:08:59.456946+0000","last_undegraded":"2024-08-16T19:08:59.456946+0000","last_fullsized":"2024-08-16T19:08:59.456946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454562+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:08:59.454562+0000","last_peered":"2024-08-16T19:08:59.454562+0000","last_clean":"2024-08-16T19:08:59.454562+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:08:59.454562+0000","last_undegraded":"2024-08-16T19:08:59.454562+0000","last_fullsized":"2024-08-16T19:08:59.454562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457104+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:08:59.457104+0000","last_peered":"2024-08-16T19:08:59.457104+0000","last_clean":"2024-08-16T19:08:59.457104+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:08:59.457104+0000","last_undegraded":"2024-08-16T19:08:59.457104+0000","last_fullsized":"2024-08-16T19:08:59.457104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:34.479577+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:09:34.479577+0000","last_peered":"2024-08-16T19:09:34.479577+0000","last_clean":"2024-08-16T19:09:34.479577+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:09:34.479577+0000","last_undegraded":"2024-08-16T19:09:34.479577+0000","last_fullsized":"2024-08-16T19:09:34.479577+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458506+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:08:59.458506+0000","last_peered":"2024-08-16T19:08:59.458506+0000","last_clean":"2024-08-16T19:08:59.458506+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:08:59.458506+0000","last_undegraded":"2024-08-16T19:08:59.458506+0000","last_fullsized":"2024-08-16T19:08:59.458506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456800+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:08:59.456800+0000","last_peered":"2024-08-16T19:08:59.456800+0000","last_clean":"2024-08-16T19:08:59.456800+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:08:59.456800+0000","last_undegraded":"2024-08-16T19:08:59.456800+0000","last_fullsized":"2024-08-16T19:08:59.456800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462530+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:08:59.462530+0000","last_peered":"2024-08-16T19:08:59.462530+0000","last_clean":"2024-08-16T19:08:59.462530+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:08:59.462530+0000","last_undegraded":"2024-08-16T19:08:59.462530+0000","last_fullsized":"2024-08-16T19:08:59.462530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.461546+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:08:59.461546+0000","last_peered":"2024-08-16T19:08:59.461546+0000","last_clean":"2024-08-16T19:08:59.461546+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:08:59.461546+0000","last_undegraded":"2024-08-16T19:08:59.461546+0000","last_fullsized":"2024-08-16T19:08:59.461546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456328+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:08:59.456328+0000","last_peered":"2024-08-16T19:08:59.456328+0000","last_clean":"2024-08-16T19:08:59.456328+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:08:59.456328+0000","last_undegraded":"2024-08-16T19:08:59.456328+0000","last_fullsized":"2024-08-16T19:08:59.456328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457966+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:08:59.457966+0000","last_peered":"2024-08-16T19:08:59.457966+0000","last_clean":"2024-08-16T19:08:59.457966+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:08:59.457966+0000","last_undegraded":"2024-08-16T19:08:59.457966+0000","last_fullsized":"2024-08-16T19:08:59.457966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455554+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:08:59.455554+0000","last_peered":"2024-08-16T19:08:59.455554+0000","last_clean":"2024-08-16T19:08:59.455554+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:08:59.455554+0000","last_undegraded":"2024-08-16T19:08:59.455554+0000","last_fullsized":"2024-08-16T19:08:59.455554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462582+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:08:59.462582+0000","last_peered":"2024-08-16T19:08:59.462582+0000","last_clean":"2024-08-16T19:08:59.462582+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:08:59.462582+0000","last_undegraded":"2024-08-16T19:08:59.462582+0000","last_fullsized":"2024-08-16T19:08:59.462582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456804+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:08:59.456804+0000","last_peered":"2024-08-16T19:08:59.456804+0000","last_clean":"2024-08-16T19:08:59.456804+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:08:59.456804+0000","last_undegraded":"2024-08-16T19:08:59.456804+0000","last_fullsized":"2024-08-16T19:08:59.456804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593293+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:08:59.593293+0000","last_peered":"2024-08-16T19:08:59.593293+0000","last_clean":"2024-08-16T19:08:59.593293+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:08:59.593293+0000","last_undegraded":"2024-08-16T19:08:59.593293+0000","last_fullsized":"2024-08-16T19:08:59.593293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595085+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:08:59.595085+0000","last_peered":"2024-08-16T19:08:59.595085+0000","last_clean":"2024-08-16T19:08:59.595085+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:08:59.595085+0000","last_undegraded":"2024-08-16T19:08:59.595085+0000","last_fullsized":"2024-08-16T19:08:59.595085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593011+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:08:59.593011+0000","last_peered":"2024-08-16T19:08:59.593011+0000","last_clean":"2024-08-16T19:08:59.593011+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:08:59.593011+0000","last_undegraded":"2024-08-16T19:08:59.593011+0000","last_fullsized":"2024-08-16T19:08:59.593011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545609+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:08:59.545609+0000","last_peered":"2024-08-16T19:08:59.545609+0000","last_clean":"2024-08-16T19:08:59.545609+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:08:59.545609+0000","last_undegraded":"2024-08-16T19:08:59.545609+0000","last_fullsized":"2024-08-16T19:08:59.545609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455713+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:08:59.455713+0000","last_peered":"2024-08-16T19:08:59.455713+0000","last_clean":"2024-08-16T19:08:59.455713+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:08:59.455713+0000","last_undegraded":"2024-08-16T19:08:59.455713+0000","last_fullsized":"2024-08-16T19:08:59.455713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466874+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:08:59.466874+0000","last_peered":"2024-08-16T19:08:59.466874+0000","last_clean":"2024-08-16T19:08:59.466874+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:08:59.466874+0000","last_undegraded":"2024-08-16T19:08:59.466874+0000","last_fullsized":"2024-08-16T19:08:59.466874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456999+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:08:59.456999+0000","last_peered":"2024-08-16T19:08:59.456999+0000","last_clean":"2024-08-16T19:08:59.456999+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:08:59.456999+0000","last_undegraded":"2024-08-16T19:08:59.456999+0000","last_fullsized":"2024-08-16T19:08:59.456999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462773+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:08:59.462773+0000","last_peered":"2024-08-16T19:08:59.462773+0000","last_clean":"2024-08-16T19:08:59.462773+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:08:59.462773+0000","last_undegraded":"2024-08-16T19:08:59.462773+0000","last_fullsized":"2024-08-16T19:08:59.462773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545175+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:08:59.545175+0000","last_peered":"2024-08-16T19:08:59.545175+0000","last_clean":"2024-08-16T19:08:59.545175+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:08:59.545175+0000","last_undegraded":"2024-08-16T19:08:59.545175+0000","last_fullsized":"2024-08-16T19:08:59.545175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457436+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:08:59.457436+0000","last_peered":"2024-08-16T19:08:59.457436+0000","last_clean":"2024-08-16T19:08:59.457436+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:08:59.457436+0000","last_undegraded":"2024-08-16T19:08:59.457436+0000","last_fullsized":"2024-08-16T19:08:59.457436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457698+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:08:59.457698+0000","last_peered":"2024-08-16T19:08:59.457698+0000","last_clean":"2024-08-16T19:08:59.457698+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:08:59.457698+0000","last_undegraded":"2024-08-16T19:08:59.457698+0000","last_fullsized":"2024-08-16T19:08:59.457698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455841+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:08:59.455841+0000","last_peered":"2024-08-16T19:08:59.455841+0000","last_clean":"2024-08-16T19:08:59.455841+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:08:59.455841+0000","last_undegraded":"2024-08-16T19:08:59.455841+0000","last_fullsized":"2024-08-16T19:08:59.455841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457704+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:08:59.457704+0000","last_peered":"2024-08-16T19:08:59.457704+0000","last_clean":"2024-08-16T19:08:59.457704+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:08:59.457704+0000","last_undegraded":"2024-08-16T19:08:59.457704+0000","last_fullsized":"2024-08-16T19:08:59.457704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455744+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:08:59.455744+0000","last_peered":"2024-08-16T19:08:59.455744+0000","last_clean":"2024-08-16T19:08:59.455744+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:08:59.455744+0000","last_undegraded":"2024-08-16T19:08:59.455744+0000","last_fullsized":"2024-08-16T19:08:59.455744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457744+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:08:59.457744+0000","last_peered":"2024-08-16T19:08:59.457744+0000","last_clean":"2024-08-16T19:08:59.457744+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:08:59.457744+0000","last_undegraded":"2024-08-16T19:08:59.457744+0000","last_fullsized":"2024-08-16T19:08:59.457744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546015+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:08:59.546015+0000","last_peered":"2024-08-16T19:08:59.546015+0000","last_clean":"2024-08-16T19:08:59.546015+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:08:59.546015+0000","last_undegraded":"2024-08-16T19:08:59.546015+0000","last_fullsized":"2024-08-16T19:08:59.546015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593295+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:08:59.593295+0000","last_peered":"2024-08-16T19:08:59.593295+0000","last_clean":"2024-08-16T19:08:59.593295+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:08:59.593295+0000","last_undegraded":"2024-08-16T19:08:59.593295+0000","last_fullsized":"2024-08-16T19:08:59.593295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454659+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:08:59.454659+0000","last_peered":"2024-08-16T19:08:59.454659+0000","last_clean":"2024-08-16T19:08:59.454659+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:08:59.454659+0000","last_undegraded":"2024-08-16T19:08:59.454659+0000","last_fullsized":"2024-08-16T19:08:59.454659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456837+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:08:59.456837+0000","last_peered":"2024-08-16T19:08:59.456837+0000","last_clean":"2024-08-16T19:08:59.456837+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:08:59.456837+0000","last_undegraded":"2024-08-16T19:08:59.456837+0000","last_fullsized":"2024-08-16T19:08:59.456837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458460+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:08:59.458460+0000","last_peered":"2024-08-16T19:08:59.458460+0000","last_clean":"2024-08-16T19:08:59.458460+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:08:59.458460+0000","last_undegraded":"2024-08-16T19:08:59.458460+0000","last_fullsized":"2024-08-16T19:08:59.458460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457358+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:08:59.457358+0000","last_peered":"2024-08-16T19:08:59.457358+0000","last_clean":"2024-08-16T19:08:59.457358+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:08:59.457358+0000","last_undegraded":"2024-08-16T19:08:59.457358+0000","last_fullsized":"2024-08-16T19:08:59.457358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458036+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:08:59.458036+0000","last_peered":"2024-08-16T19:08:59.458036+0000","last_clean":"2024-08-16T19:08:59.458036+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:08:59.458036+0000","last_undegraded":"2024-08-16T19:08:59.458036+0000","last_fullsized":"2024-08-16T19:08:59.458036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457901+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:08:59.457901+0000","last_peered":"2024-08-16T19:08:59.457901+0000","last_clean":"2024-08-16T19:08:59.457901+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:08:59.457901+0000","last_undegraded":"2024-08-16T19:08:59.457901+0000","last_fullsized":"2024-08-16T19:08:59.457901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462449+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:08:59.462449+0000","last_peered":"2024-08-16T19:08:59.462449+0000","last_clean":"2024-08-16T19:08:59.462449+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:08:59.462449+0000","last_undegraded":"2024-08-16T19:08:59.462449+0000","last_fullsized":"2024-08-16T19:08:59.462449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":537,"num_read_kb":349,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1724416,"data_stored":1157894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939689,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":512644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":508926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1105812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":504442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1100614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":508516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":503760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":567762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":567542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":505898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:09:40.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:09:40.627 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 -- 172.21.15.62:0/3090493524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a00101150 msgr2=0x7f2a0010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/3090493524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a00101150 0x7f2a0010d680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009cd0 tx=0x7f29f802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 -- 172.21.15.62:0/3090493524 shutdown_connections 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/3090493524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a00101150 0x7f2a0010d680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/3090493524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001007b0 0x7f2a00100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/3090493524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a001067b0 0x7f2a00106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 -- 172.21.15.62:0/3090493524 >> 172.21.15.62:0/3090493524 conn(0x7f2a000fc4d0 msgr2=0x7f2a000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 -- 172.21.15.62:0/3090493524 shutdown_connections 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.093+0000 7f2a05a8e640 1 -- 172.21.15.62:0/3090493524 wait complete. 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 Processor -- start 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 -- start start 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a00101150 0x7f2a001986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 0x7f2a0019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110bb0 con 0x7f2a001007b0 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110a30 con 0x7f2a001067b0 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.094+0000 7f2a05a8e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110d30 con 0x7f2a00101150 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:41.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42060/0 (socket says 172.21.15.62:42060) 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 -- 172.21.15.62:0/1405412949 learned_addr learned my addr 172.21.15.62:0/1405412949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29feffd640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a00101150 0x7f2a001986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29fffff640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 0x7f2a0019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 -- 172.21.15.62:0/1405412949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a00101150 msgr2=0x7f2a001986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a00101150 0x7f2a001986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 -- 172.21.15.62:0/1405412949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 msgr2=0x7f2a0019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 0x7f2a0019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 -- 172.21.15.62:0/1405412949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0019d140 con 0x7f2a001007b0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29fffff640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 0x7f2a0019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29ff7fe640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f29f000d770 tx=0x7f29f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0017070 con 0x7f2a001007b0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f0005500 con 0x7f2a001007b0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0019d430 con 0x7f2a001007b0 2024-08-16T19:09:41.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0004e00 con 0x7f2a001007b0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.095+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a001a4d10 con 0x7f2a001007b0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.096+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00100c10 con 0x7f2a001007b0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.098+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f29f0004fa0 con 0x7f2a001007b0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.099+0000 7f29fcff9640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 0x7f29e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.099+0000 7f29feffd640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 0x7f29e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.099+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f29f00cf370 con 0x7f2a001007b0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.099+0000 7f29feffd640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 0x7f29e80a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a00104f30 tx=0x7f29f400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:41.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.102+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f29f009be30 con 0x7f2a001007b0 2024-08-16T19:09:41.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.275+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a00062360 con 0x7f29e80a2c90 2024-08-16T19:09:41.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.284+0000 7f29fcff9640 1 -- 172.21.15.62:0/1405412949 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340404 (secure 0 0 0) 0x7f2a00062360 con 0x7f29e80a2c90 2024-08-16T19:09:41.289 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:41.290 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 msgr2=0x7f29e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 0x7f29e80a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a00104f30 tx=0x7f29f400a2d0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 msgr2=0x7f2a00198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f29f000d770 tx=0x7f29f000dc30 comp rx=0 tx=0).stop 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 shutdown_connections 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f29e80a2c90 0x7f29e80a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a001067b0 0x7f2a0019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a00101150 0x7f2a001986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 --2- 172.21.15.62:0/1405412949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a001007b0 0x7f2a00198190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 >> 172.21.15.62:0/1405412949 conn(0x7f2a000fc4d0 msgr2=0x7f2a0006a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 shutdown_connections 2024-08-16T19:09:41.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:41.288+0000 7f2a05a8e640 1 -- 172.21.15.62:0/1405412949 wait complete. 2024-08-16T19:09:41.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:41 smithi062 ceph-mon[32993]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:41.609 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:41 smithi062 ceph-mon[32993]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:41 smithi155 ceph-mon[37413]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:41 smithi155 ceph-mon[37413]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:41 smithi161 ceph-mon[37576]: pgmap v311: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:41 smithi161 ceph-mon[37576]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:41.915 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-08-16T19:09:39.670532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660264,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":673,"num_read_kb":468,"num_write":432,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375920,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565904,"statfs":{"total":1151940427776,"available":1151555485696,"internally_reserved":0,"allocated":14569472,"data_stored":8052818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003444"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.487171+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:09:33.487171+0000","last_peered":"2024-08-16T19:09:33.487171+0000","last_clean":"2024-08-16T19:09:33.487171+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:09:33.487171+0000","last_undegraded":"2024-08-16T19:09:33.487171+0000","last_fullsized":"2024-08-16T19:09:33.487171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592994+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:08:59.592994+0000","last_peered":"2024-08-16T19:08:59.592994+0000","last_clean":"2024-08-16T19:08:59.592994+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:08:59.592994+0000","last_undegraded":"2024-08-16T19:08:59.592994+0000","last_fullsized":"2024-08-16T19:08:59.592994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456279+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:08:59.456279+0000","last_peered":"2024-08-16T19:08:59.456279+0000","last_clean":"2024-08-16T19:08:59.456279+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:08:59.456279+0000","last_undegraded":"2024-08-16T19:08:59.456279+0000","last_fullsized":"2024-08-16T19:08:59.456279+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593258+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:08:59.593258+0000","last_peered":"2024-08-16T19:08:59.593258+0000","last_clean":"2024-08-16T19:08:59.593258+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:08:59.593258+0000","last_undegraded":"2024-08-16T19:08:59.593258+0000","last_fullsized":"2024-08-16T19:08:59.593258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458615+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:08:59.458615+0000","last_peered":"2024-08-16T19:08:59.458615+0000","last_clean":"2024-08-16T19:08:59.458615+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:08:59.458615+0000","last_undegraded":"2024-08-16T19:08:59.458615+0000","last_fullsized":"2024-08-16T19:08:59.458615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.594556+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:08:59.594556+0000","last_peered":"2024-08-16T19:08:59.594556+0000","last_clean":"2024-08-16T19:08:59.594556+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:08:59.594556+0000","last_undegraded":"2024-08-16T19:08:59.594556+0000","last_fullsized":"2024-08-16T19:08:59.594556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459473+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:08:59.459473+0000","last_peered":"2024-08-16T19:08:59.459473+0000","last_clean":"2024-08-16T19:08:59.459473+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:08:59.459473+0000","last_undegraded":"2024-08-16T19:08:59.459473+0000","last_fullsized":"2024-08-16T19:08:59.459473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545800+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:08:59.545800+0000","last_peered":"2024-08-16T19:08:59.545800+0000","last_clean":"2024-08-16T19:08:59.545800+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:08:59.545800+0000","last_undegraded":"2024-08-16T19:08:59.545800+0000","last_fullsized":"2024-08-16T19:08:59.545800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457100+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:08:59.457100+0000","last_peered":"2024-08-16T19:08:59.457100+0000","last_clean":"2024-08-16T19:08:59.457100+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:08:59.457100+0000","last_undegraded":"2024-08-16T19:08:59.457100+0000","last_fullsized":"2024-08-16T19:08:59.457100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459443+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:08:59.459443+0000","last_peered":"2024-08-16T19:08:59.459443+0000","last_clean":"2024-08-16T19:08:59.459443+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:08:59.459443+0000","last_undegraded":"2024-08-16T19:08:59.459443+0000","last_fullsized":"2024-08-16T19:08:59.459443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.547859+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:08:59.547859+0000","last_peered":"2024-08-16T19:08:59.547859+0000","last_clean":"2024-08-16T19:08:59.547859+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:08:59.547859+0000","last_undegraded":"2024-08-16T19:08:59.547859+0000","last_fullsized":"2024-08-16T19:08:59.547859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457189+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:08:59.457189+0000","last_peered":"2024-08-16T19:08:59.457189+0000","last_clean":"2024-08-16T19:08:59.457189+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:08:59.457189+0000","last_undegraded":"2024-08-16T19:08:59.457189+0000","last_fullsized":"2024-08-16T19:08:59.457189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454902+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:08:59.454902+0000","last_peered":"2024-08-16T19:08:59.454902+0000","last_clean":"2024-08-16T19:08:59.454902+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:08:59.454902+0000","last_undegraded":"2024-08-16T19:08:59.454902+0000","last_fullsized":"2024-08-16T19:08:59.454902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458666+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:08:59.458666+0000","last_peered":"2024-08-16T19:08:59.458666+0000","last_clean":"2024-08-16T19:08:59.458666+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:08:59.458666+0000","last_undegraded":"2024-08-16T19:08:59.458666+0000","last_fullsized":"2024-08-16T19:08:59.458666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457164+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:08:59.457164+0000","last_peered":"2024-08-16T19:08:59.457164+0000","last_clean":"2024-08-16T19:08:59.457164+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:08:59.457164+0000","last_undegraded":"2024-08-16T19:08:59.457164+0000","last_fullsized":"2024-08-16T19:08:59.457164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592793+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:08:59.592793+0000","last_peered":"2024-08-16T19:08:59.592793+0000","last_clean":"2024-08-16T19:08:59.592793+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:08:59.592793+0000","last_undegraded":"2024-08-16T19:08:59.592793+0000","last_fullsized":"2024-08-16T19:08:59.592793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456614+0000","last_change":"2024-08-16T19:08:02.462372+0000","last_active":"2024-08-16T19:08:59.456614+0000","last_peered":"2024-08-16T19:08:59.456614+0000","last_clean":"2024-08-16T19:08:59.456614+0000","last_became_active":"2024-08-16T19:08:02.461973+0000","last_became_peered":"2024-08-16T19:08:02.461973+0000","last_unstale":"2024-08-16T19:08:59.456614+0000","last_undegraded":"2024-08-16T19:08:59.456614+0000","last_fullsized":"2024-08-16T19:08:59.456614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:24:04.191300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457135+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:08:59.457135+0000","last_peered":"2024-08-16T19:08:59.457135+0000","last_clean":"2024-08-16T19:08:59.457135+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:08:59.457135+0000","last_undegraded":"2024-08-16T19:08:59.457135+0000","last_fullsized":"2024-08-16T19:08:59.457135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462630+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:08:59.462630+0000","last_peered":"2024-08-16T19:08:59.462630+0000","last_clean":"2024-08-16T19:08:59.462630+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:08:59.462630+0000","last_undegraded":"2024-08-16T19:08:59.462630+0000","last_fullsized":"2024-08-16T19:08:59.462630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593234+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:08:59.593234+0000","last_peered":"2024-08-16T19:08:59.593234+0000","last_clean":"2024-08-16T19:08:59.593234+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:08:59.593234+0000","last_undegraded":"2024-08-16T19:08:59.593234+0000","last_fullsized":"2024-08-16T19:08:59.593234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457650+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:08:59.457650+0000","last_peered":"2024-08-16T19:08:59.457650+0000","last_clean":"2024-08-16T19:08:59.457650+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:08:59.457650+0000","last_undegraded":"2024-08-16T19:08:59.457650+0000","last_fullsized":"2024-08-16T19:08:59.457650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458319+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:08:59.458319+0000","last_peered":"2024-08-16T19:08:59.458319+0000","last_clean":"2024-08-16T19:08:59.458319+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:08:59.458319+0000","last_undegraded":"2024-08-16T19:08:59.458319+0000","last_fullsized":"2024-08-16T19:08:59.458319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545351+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:08:59.545351+0000","last_peered":"2024-08-16T19:08:59.545351+0000","last_clean":"2024-08-16T19:08:59.545351+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:08:59.545351+0000","last_undegraded":"2024-08-16T19:08:59.545351+0000","last_fullsized":"2024-08-16T19:08:59.545351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.464030+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:08:59.464030+0000","last_peered":"2024-08-16T19:08:59.464030+0000","last_clean":"2024-08-16T19:08:59.464030+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:08:59.464030+0000","last_undegraded":"2024-08-16T19:08:59.464030+0000","last_fullsized":"2024-08-16T19:08:59.464030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458222+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:08:59.458222+0000","last_peered":"2024-08-16T19:08:59.458222+0000","last_clean":"2024-08-16T19:08:59.458222+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:08:59.458222+0000","last_undegraded":"2024-08-16T19:08:59.458222+0000","last_fullsized":"2024-08-16T19:08:59.458222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458400+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:08:59.458400+0000","last_peered":"2024-08-16T19:08:59.458400+0000","last_clean":"2024-08-16T19:08:59.458400+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:08:59.458400+0000","last_undegraded":"2024-08-16T19:08:59.458400+0000","last_fullsized":"2024-08-16T19:08:59.458400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456408+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:08:59.456408+0000","last_peered":"2024-08-16T19:08:59.456408+0000","last_clean":"2024-08-16T19:08:59.456408+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:08:59.456408+0000","last_undegraded":"2024-08-16T19:08:59.456408+0000","last_fullsized":"2024-08-16T19:08:59.456408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466015+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:08:59.466015+0000","last_peered":"2024-08-16T19:08:59.466015+0000","last_clean":"2024-08-16T19:08:59.466015+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:08:59.466015+0000","last_undegraded":"2024-08-16T19:08:59.466015+0000","last_fullsized":"2024-08-16T19:08:59.466015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:00.591976+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:09:00.591976+0000","last_peered":"2024-08-16T19:09:00.591976+0000","last_clean":"2024-08-16T19:09:00.591976+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:09:00.591976+0000","last_undegraded":"2024-08-16T19:09:00.591976+0000","last_fullsized":"2024-08-16T19:09:00.591976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593074+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:08:59.593074+0000","last_peered":"2024-08-16T19:08:59.593074+0000","last_clean":"2024-08-16T19:08:59.593074+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:08:59.593074+0000","last_undegraded":"2024-08-16T19:08:59.593074+0000","last_fullsized":"2024-08-16T19:08:59.593074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455577+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:08:59.455577+0000","last_peered":"2024-08-16T19:08:59.455577+0000","last_clean":"2024-08-16T19:08:59.455577+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:08:59.455577+0000","last_undegraded":"2024-08-16T19:08:59.455577+0000","last_fullsized":"2024-08-16T19:08:59.455577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459679+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:08:59.459679+0000","last_peered":"2024-08-16T19:08:59.459679+0000","last_clean":"2024-08-16T19:08:59.459679+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:08:59.459679+0000","last_undegraded":"2024-08-16T19:08:59.459679+0000","last_fullsized":"2024-08-16T19:08:59.459679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462351+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:08:59.462351+0000","last_peered":"2024-08-16T19:08:59.462351+0000","last_clean":"2024-08-16T19:08:59.462351+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:08:59.462351+0000","last_undegraded":"2024-08-16T19:08:59.462351+0000","last_fullsized":"2024-08-16T19:08:59.462351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458157+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:08:59.458157+0000","last_peered":"2024-08-16T19:08:59.458157+0000","last_clean":"2024-08-16T19:08:59.458157+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:08:59.458157+0000","last_undegraded":"2024-08-16T19:08:59.458157+0000","last_fullsized":"2024-08-16T19:08:59.458157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456378+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:08:59.456378+0000","last_peered":"2024-08-16T19:08:59.456378+0000","last_clean":"2024-08-16T19:08:59.456378+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:08:59.456378+0000","last_undegraded":"2024-08-16T19:08:59.456378+0000","last_fullsized":"2024-08-16T19:08:59.456378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458335+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:08:59.458335+0000","last_peered":"2024-08-16T19:08:59.458335+0000","last_clean":"2024-08-16T19:08:59.458335+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:08:59.458335+0000","last_undegraded":"2024-08-16T19:08:59.458335+0000","last_fullsized":"2024-08-16T19:08:59.458335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457766+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:08:59.457766+0000","last_peered":"2024-08-16T19:08:59.457766+0000","last_clean":"2024-08-16T19:08:59.457766+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:08:59.457766+0000","last_undegraded":"2024-08-16T19:08:59.457766+0000","last_fullsized":"2024-08-16T19:08:59.457766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.487635+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:09:33.487635+0000","last_peered":"2024-08-16T19:09:33.487635+0000","last_clean":"2024-08-16T19:09:33.487635+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:09:33.487635+0000","last_undegraded":"2024-08-16T19:09:33.487635+0000","last_fullsized":"2024-08-16T19:09:33.487635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455413+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:08:59.455413+0000","last_peered":"2024-08-16T19:08:59.455413+0000","last_clean":"2024-08-16T19:08:59.455413+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:08:59.455413+0000","last_undegraded":"2024-08-16T19:08:59.455413+0000","last_fullsized":"2024-08-16T19:08:59.455413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454239+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:08:59.454239+0000","last_peered":"2024-08-16T19:08:59.454239+0000","last_clean":"2024-08-16T19:08:59.454239+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:08:59.454239+0000","last_undegraded":"2024-08-16T19:08:59.454239+0000","last_fullsized":"2024-08-16T19:08:59.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.481408+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:09:36.481408+0000","last_peered":"2024-08-16T19:09:36.481408+0000","last_clean":"2024-08-16T19:09:36.481408+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:09:36.481408+0000","last_undegraded":"2024-08-16T19:09:36.481408+0000","last_fullsized":"2024-08-16T19:09:36.481408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455963+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:08:59.455963+0000","last_peered":"2024-08-16T19:08:59.455963+0000","last_clean":"2024-08-16T19:08:59.455963+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:08:59.455963+0000","last_undegraded":"2024-08-16T19:08:59.455963+0000","last_fullsized":"2024-08-16T19:08:59.455963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455170+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:08:59.455170+0000","last_peered":"2024-08-16T19:08:59.455170+0000","last_clean":"2024-08-16T19:08:59.455170+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:08:59.455170+0000","last_undegraded":"2024-08-16T19:08:59.455170+0000","last_fullsized":"2024-08-16T19:08:59.455170+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454875+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:08:59.454875+0000","last_peered":"2024-08-16T19:08:59.454875+0000","last_clean":"2024-08-16T19:08:59.454875+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:08:59.454875+0000","last_undegraded":"2024-08-16T19:08:59.454875+0000","last_fullsized":"2024-08-16T19:08:59.454875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454959+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:08:59.454959+0000","last_peered":"2024-08-16T19:08:59.454959+0000","last_clean":"2024-08-16T19:08:59.454959+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:08:59.454959+0000","last_undegraded":"2024-08-16T19:08:59.454959+0000","last_fullsized":"2024-08-16T19:08:59.454959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592928+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:08:59.592928+0000","last_peered":"2024-08-16T19:08:59.592928+0000","last_clean":"2024-08-16T19:08:59.592928+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:08:59.592928+0000","last_undegraded":"2024-08-16T19:08:59.592928+0000","last_fullsized":"2024-08-16T19:08:59.592928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458401+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:08:59.458401+0000","last_peered":"2024-08-16T19:08:59.458401+0000","last_clean":"2024-08-16T19:08:59.458401+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:08:59.458401+0000","last_undegraded":"2024-08-16T19:08:59.458401+0000","last_fullsized":"2024-08-16T19:08:59.458401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455187+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:08:59.455187+0000","last_peered":"2024-08-16T19:08:59.455187+0000","last_clean":"2024-08-16T19:08:59.455187+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:08:59.455187+0000","last_undegraded":"2024-08-16T19:08:59.455187+0000","last_fullsized":"2024-08-16T19:08:59.455187+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455400+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:08:59.455400+0000","last_peered":"2024-08-16T19:08:59.455400+0000","last_clean":"2024-08-16T19:08:59.455400+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:08:59.455400+0000","last_undegraded":"2024-08-16T19:08:59.455400+0000","last_fullsized":"2024-08-16T19:08:59.455400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455750+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:08:59.455750+0000","last_peered":"2024-08-16T19:08:59.455750+0000","last_clean":"2024-08-16T19:08:59.455750+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:08:59.455750+0000","last_undegraded":"2024-08-16T19:08:59.455750+0000","last_fullsized":"2024-08-16T19:08:59.455750+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:35.494168+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:09:35.494168+0000","last_peered":"2024-08-16T19:09:35.494168+0000","last_clean":"2024-08-16T19:09:35.494168+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:09:35.494168+0000","last_undegraded":"2024-08-16T19:09:35.494168+0000","last_fullsized":"2024-08-16T19:09:35.494168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545422+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:08:59.545422+0000","last_peered":"2024-08-16T19:08:59.545422+0000","last_clean":"2024-08-16T19:08:59.545422+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:08:59.545422+0000","last_undegraded":"2024-08-16T19:08:59.545422+0000","last_fullsized":"2024-08-16T19:08:59.545422+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455331+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:08:59.455331+0000","last_peered":"2024-08-16T19:08:59.455331+0000","last_clean":"2024-08-16T19:08:59.455331+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:08:59.455331+0000","last_undegraded":"2024-08-16T19:08:59.455331+0000","last_fullsized":"2024-08-16T19:08:59.455331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454825+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:08:59.454825+0000","last_peered":"2024-08-16T19:08:59.454825+0000","last_clean":"2024-08-16T19:08:59.454825+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:08:59.454825+0000","last_undegraded":"2024-08-16T19:08:59.454825+0000","last_fullsized":"2024-08-16T19:08:59.454825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457861+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:08:59.457861+0000","last_peered":"2024-08-16T19:08:59.457861+0000","last_clean":"2024-08-16T19:08:59.457861+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:08:59.457861+0000","last_undegraded":"2024-08-16T19:08:59.457861+0000","last_fullsized":"2024-08-16T19:08:59.457861+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455492+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:08:59.455492+0000","last_peered":"2024-08-16T19:08:59.455492+0000","last_clean":"2024-08-16T19:08:59.455492+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:08:59.455492+0000","last_undegraded":"2024-08-16T19:08:59.455492+0000","last_fullsized":"2024-08-16T19:08:59.455492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455979+0000","last_change":"2024-08-16T19:08:02.457345+0000","last_active":"2024-08-16T19:08:59.455979+0000","last_peered":"2024-08-16T19:08:59.455979+0000","last_clean":"2024-08-16T19:08:59.455979+0000","last_became_active":"2024-08-16T19:08:02.456961+0000","last_became_peered":"2024-08-16T19:08:02.456961+0000","last_unstale":"2024-08-16T19:08:59.455979+0000","last_undegraded":"2024-08-16T19:08:59.455979+0000","last_fullsized":"2024-08-16T19:08:59.455979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:24.143886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454067+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:08:59.454067+0000","last_peered":"2024-08-16T19:08:59.454067+0000","last_clean":"2024-08-16T19:08:59.454067+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:08:59.454067+0000","last_undegraded":"2024-08-16T19:08:59.454067+0000","last_fullsized":"2024-08-16T19:08:59.454067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462353+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:08:59.462353+0000","last_peered":"2024-08-16T19:08:59.462353+0000","last_clean":"2024-08-16T19:08:59.462353+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:08:59.462353+0000","last_undegraded":"2024-08-16T19:08:59.462353+0000","last_fullsized":"2024-08-16T19:08:59.462353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459864+0000","last_change":"2024-08-16T19:08:02.457999+0000","last_active":"2024-08-16T19:08:59.459864+0000","last_peered":"2024-08-16T19:08:59.459864+0000","last_clean":"2024-08-16T19:08:59.459864+0000","last_became_active":"2024-08-16T19:08:02.456740+0000","last_became_peered":"2024-08-16T19:08:02.456740+0000","last_unstale":"2024-08-16T19:08:59.459864+0000","last_undegraded":"2024-08-16T19:08:59.459864+0000","last_fullsized":"2024-08-16T19:08:59.459864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:20:22.707081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454151+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:08:59.454151+0000","last_peered":"2024-08-16T19:08:59.454151+0000","last_clean":"2024-08-16T19:08:59.454151+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:08:59.454151+0000","last_undegraded":"2024-08-16T19:08:59.454151+0000","last_fullsized":"2024-08-16T19:08:59.454151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454277+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:08:59.454277+0000","last_peered":"2024-08-16T19:08:59.454277+0000","last_clean":"2024-08-16T19:08:59.454277+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:08:59.454277+0000","last_undegraded":"2024-08-16T19:08:59.454277+0000","last_fullsized":"2024-08-16T19:08:59.454277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458557+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:08:59.458557+0000","last_peered":"2024-08-16T19:08:59.458557+0000","last_clean":"2024-08-16T19:08:59.458557+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:08:59.458557+0000","last_undegraded":"2024-08-16T19:08:59.458557+0000","last_fullsized":"2024-08-16T19:08:59.458557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455944+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:08:59.455944+0000","last_peered":"2024-08-16T19:08:59.455944+0000","last_clean":"2024-08-16T19:08:59.455944+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:08:59.455944+0000","last_undegraded":"2024-08-16T19:08:59.455944+0000","last_fullsized":"2024-08-16T19:08:59.455944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454634+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:08:59.454634+0000","last_peered":"2024-08-16T19:08:59.454634+0000","last_clean":"2024-08-16T19:08:59.454634+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:08:59.454634+0000","last_undegraded":"2024-08-16T19:08:59.454634+0000","last_fullsized":"2024-08-16T19:08:59.454634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:33.477295+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:09:33.477295+0000","last_peered":"2024-08-16T19:09:33.477295+0000","last_clean":"2024-08-16T19:09:33.477295+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:09:33.477295+0000","last_undegraded":"2024-08-16T19:09:33.477295+0000","last_fullsized":"2024-08-16T19:09:33.477295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595244+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:08:59.595244+0000","last_peered":"2024-08-16T19:08:59.595244+0000","last_clean":"2024-08-16T19:08:59.595244+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:08:59.595244+0000","last_undegraded":"2024-08-16T19:08:59.595244+0000","last_fullsized":"2024-08-16T19:08:59.595244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455402+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:08:59.455402+0000","last_peered":"2024-08-16T19:08:59.455402+0000","last_clean":"2024-08-16T19:08:59.455402+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:08:59.455402+0000","last_undegraded":"2024-08-16T19:08:59.455402+0000","last_fullsized":"2024-08-16T19:08:59.455402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454397+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:08:59.454397+0000","last_peered":"2024-08-16T19:08:59.454397+0000","last_clean":"2024-08-16T19:08:59.454397+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:08:59.454397+0000","last_undegraded":"2024-08-16T19:08:59.454397+0000","last_fullsized":"2024-08-16T19:08:59.454397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462052+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:08:59.462052+0000","last_peered":"2024-08-16T19:08:59.462052+0000","last_clean":"2024-08-16T19:08:59.462052+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:08:59.462052+0000","last_undegraded":"2024-08-16T19:08:59.462052+0000","last_fullsized":"2024-08-16T19:08:59.462052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455923+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:08:59.455923+0000","last_peered":"2024-08-16T19:08:59.455923+0000","last_clean":"2024-08-16T19:08:59.455923+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:08:59.455923+0000","last_undegraded":"2024-08-16T19:08:59.455923+0000","last_fullsized":"2024-08-16T19:08:59.455923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595109+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:08:59.595109+0000","last_peered":"2024-08-16T19:08:59.595109+0000","last_clean":"2024-08-16T19:08:59.595109+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:08:59.595109+0000","last_undegraded":"2024-08-16T19:08:59.595109+0000","last_fullsized":"2024-08-16T19:08:59.595109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.459318+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:08:59.459318+0000","last_peered":"2024-08-16T19:08:59.459318+0000","last_clean":"2024-08-16T19:08:59.459318+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:08:59.459318+0000","last_undegraded":"2024-08-16T19:08:59.459318+0000","last_fullsized":"2024-08-16T19:08:59.459318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":178,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.486587+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:09:36.486587+0000","last_peered":"2024-08-16T19:09:36.486587+0000","last_clean":"2024-08-16T19:09:36.486587+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:09:36.486587+0000","last_undegraded":"2024-08-16T19:09:36.486587+0000","last_fullsized":"2024-08-16T19:09:36.486587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62070,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":117,"num_write":36,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456560+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:08:59.456560+0000","last_peered":"2024-08-16T19:08:59.456560+0000","last_clean":"2024-08-16T19:08:59.456560+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:08:59.456560+0000","last_undegraded":"2024-08-16T19:08:59.456560+0000","last_fullsized":"2024-08-16T19:08:59.456560+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546106+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:08:59.546106+0000","last_peered":"2024-08-16T19:08:59.546106+0000","last_clean":"2024-08-16T19:08:59.546106+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:08:59.546106+0000","last_undegraded":"2024-08-16T19:08:59.546106+0000","last_fullsized":"2024-08-16T19:08:59.546106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454804+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:08:59.454804+0000","last_peered":"2024-08-16T19:08:59.454804+0000","last_clean":"2024-08-16T19:08:59.454804+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:08:59.454804+0000","last_undegraded":"2024-08-16T19:08:59.454804+0000","last_fullsized":"2024-08-16T19:08:59.454804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458095+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:08:59.458095+0000","last_peered":"2024-08-16T19:08:59.458095+0000","last_clean":"2024-08-16T19:08:59.458095+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:08:59.458095+0000","last_undegraded":"2024-08-16T19:08:59.458095+0000","last_fullsized":"2024-08-16T19:08:59.458095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.460897+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:08:59.460897+0000","last_peered":"2024-08-16T19:08:59.460897+0000","last_clean":"2024-08-16T19:08:59.460897+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:08:59.460897+0000","last_undegraded":"2024-08-16T19:08:59.460897+0000","last_fullsized":"2024-08-16T19:08:59.460897+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455542+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:08:59.455542+0000","last_peered":"2024-08-16T19:08:59.455542+0000","last_clean":"2024-08-16T19:08:59.455542+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:08:59.455542+0000","last_undegraded":"2024-08-16T19:08:59.455542+0000","last_fullsized":"2024-08-16T19:08:59.455542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456325+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:08:59.456325+0000","last_peered":"2024-08-16T19:08:59.456325+0000","last_clean":"2024-08-16T19:08:59.456325+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:08:59.456325+0000","last_undegraded":"2024-08-16T19:08:59.456325+0000","last_fullsized":"2024-08-16T19:08:59.456325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545951+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:08:59.545951+0000","last_peered":"2024-08-16T19:08:59.545951+0000","last_clean":"2024-08-16T19:08:59.545951+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:08:59.545951+0000","last_undegraded":"2024-08-16T19:08:59.545951+0000","last_fullsized":"2024-08-16T19:08:59.545951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:36.481558+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:09:36.481558+0000","last_peered":"2024-08-16T19:09:36.481558+0000","last_clean":"2024-08-16T19:09:36.481558+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:09:36.481558+0000","last_undegraded":"2024-08-16T19:09:36.481558+0000","last_fullsized":"2024-08-16T19:09:36.481558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.592679+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:08:59.592679+0000","last_peered":"2024-08-16T19:08:59.592679+0000","last_clean":"2024-08-16T19:08:59.592679+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:08:59.592679+0000","last_undegraded":"2024-08-16T19:08:59.592679+0000","last_fullsized":"2024-08-16T19:08:59.592679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457785+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:08:59.457785+0000","last_peered":"2024-08-16T19:08:59.457785+0000","last_clean":"2024-08-16T19:08:59.457785+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:08:59.457785+0000","last_undegraded":"2024-08-16T19:08:59.457785+0000","last_fullsized":"2024-08-16T19:08:59.457785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456321+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:08:59.456321+0000","last_peered":"2024-08-16T19:08:59.456321+0000","last_clean":"2024-08-16T19:08:59.456321+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:08:59.456321+0000","last_undegraded":"2024-08-16T19:08:59.456321+0000","last_fullsized":"2024-08-16T19:08:59.456321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454450+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:08:59.454450+0000","last_peered":"2024-08-16T19:08:59.454450+0000","last_clean":"2024-08-16T19:08:59.454450+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:08:59.454450+0000","last_undegraded":"2024-08-16T19:08:59.454450+0000","last_fullsized":"2024-08-16T19:08:59.454450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462131+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:08:59.462131+0000","last_peered":"2024-08-16T19:08:59.462131+0000","last_clean":"2024-08-16T19:08:59.462131+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:08:59.462131+0000","last_undegraded":"2024-08-16T19:08:59.462131+0000","last_fullsized":"2024-08-16T19:08:59.462131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593351+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:08:59.593351+0000","last_peered":"2024-08-16T19:08:59.593351+0000","last_clean":"2024-08-16T19:08:59.593351+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:08:59.593351+0000","last_undegraded":"2024-08-16T19:08:59.593351+0000","last_fullsized":"2024-08-16T19:08:59.593351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455179+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:08:59.455179+0000","last_peered":"2024-08-16T19:08:59.455179+0000","last_clean":"2024-08-16T19:08:59.455179+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:08:59.455179+0000","last_undegraded":"2024-08-16T19:08:59.455179+0000","last_fullsized":"2024-08-16T19:08:59.455179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456946+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:08:59.456946+0000","last_peered":"2024-08-16T19:08:59.456946+0000","last_clean":"2024-08-16T19:08:59.456946+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:08:59.456946+0000","last_undegraded":"2024-08-16T19:08:59.456946+0000","last_fullsized":"2024-08-16T19:08:59.456946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454562+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:08:59.454562+0000","last_peered":"2024-08-16T19:08:59.454562+0000","last_clean":"2024-08-16T19:08:59.454562+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:08:59.454562+0000","last_undegraded":"2024-08-16T19:08:59.454562+0000","last_fullsized":"2024-08-16T19:08:59.454562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457104+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:08:59.457104+0000","last_peered":"2024-08-16T19:08:59.457104+0000","last_clean":"2024-08-16T19:08:59.457104+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:08:59.457104+0000","last_undegraded":"2024-08-16T19:08:59.457104+0000","last_fullsized":"2024-08-16T19:08:59.457104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:09:34.479577+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:09:34.479577+0000","last_peered":"2024-08-16T19:09:34.479577+0000","last_clean":"2024-08-16T19:09:34.479577+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:09:34.479577+0000","last_undegraded":"2024-08-16T19:09:34.479577+0000","last_fullsized":"2024-08-16T19:09:34.479577+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458506+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:08:59.458506+0000","last_peered":"2024-08-16T19:08:59.458506+0000","last_clean":"2024-08-16T19:08:59.458506+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:08:59.458506+0000","last_undegraded":"2024-08-16T19:08:59.458506+0000","last_fullsized":"2024-08-16T19:08:59.458506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456800+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:08:59.456800+0000","last_peered":"2024-08-16T19:08:59.456800+0000","last_clean":"2024-08-16T19:08:59.456800+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:08:59.456800+0000","last_undegraded":"2024-08-16T19:08:59.456800+0000","last_fullsized":"2024-08-16T19:08:59.456800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462530+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:08:59.462530+0000","last_peered":"2024-08-16T19:08:59.462530+0000","last_clean":"2024-08-16T19:08:59.462530+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:08:59.462530+0000","last_undegraded":"2024-08-16T19:08:59.462530+0000","last_fullsized":"2024-08-16T19:08:59.462530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.461546+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:08:59.461546+0000","last_peered":"2024-08-16T19:08:59.461546+0000","last_clean":"2024-08-16T19:08:59.461546+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:08:59.461546+0000","last_undegraded":"2024-08-16T19:08:59.461546+0000","last_fullsized":"2024-08-16T19:08:59.461546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456328+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:08:59.456328+0000","last_peered":"2024-08-16T19:08:59.456328+0000","last_clean":"2024-08-16T19:08:59.456328+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:08:59.456328+0000","last_undegraded":"2024-08-16T19:08:59.456328+0000","last_fullsized":"2024-08-16T19:08:59.456328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457966+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:08:59.457966+0000","last_peered":"2024-08-16T19:08:59.457966+0000","last_clean":"2024-08-16T19:08:59.457966+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:08:59.457966+0000","last_undegraded":"2024-08-16T19:08:59.457966+0000","last_fullsized":"2024-08-16T19:08:59.457966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455554+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:08:59.455554+0000","last_peered":"2024-08-16T19:08:59.455554+0000","last_clean":"2024-08-16T19:08:59.455554+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:08:59.455554+0000","last_undegraded":"2024-08-16T19:08:59.455554+0000","last_fullsized":"2024-08-16T19:08:59.455554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462582+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:08:59.462582+0000","last_peered":"2024-08-16T19:08:59.462582+0000","last_clean":"2024-08-16T19:08:59.462582+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:08:59.462582+0000","last_undegraded":"2024-08-16T19:08:59.462582+0000","last_fullsized":"2024-08-16T19:08:59.462582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456804+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:08:59.456804+0000","last_peered":"2024-08-16T19:08:59.456804+0000","last_clean":"2024-08-16T19:08:59.456804+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:08:59.456804+0000","last_undegraded":"2024-08-16T19:08:59.456804+0000","last_fullsized":"2024-08-16T19:08:59.456804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593293+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:08:59.593293+0000","last_peered":"2024-08-16T19:08:59.593293+0000","last_clean":"2024-08-16T19:08:59.593293+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:08:59.593293+0000","last_undegraded":"2024-08-16T19:08:59.593293+0000","last_fullsized":"2024-08-16T19:08:59.593293+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.595085+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:08:59.595085+0000","last_peered":"2024-08-16T19:08:59.595085+0000","last_clean":"2024-08-16T19:08:59.595085+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:08:59.595085+0000","last_undegraded":"2024-08-16T19:08:59.595085+0000","last_fullsized":"2024-08-16T19:08:59.595085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593011+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:08:59.593011+0000","last_peered":"2024-08-16T19:08:59.593011+0000","last_clean":"2024-08-16T19:08:59.593011+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:08:59.593011+0000","last_undegraded":"2024-08-16T19:08:59.593011+0000","last_fullsized":"2024-08-16T19:08:59.593011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545609+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:08:59.545609+0000","last_peered":"2024-08-16T19:08:59.545609+0000","last_clean":"2024-08-16T19:08:59.545609+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:08:59.545609+0000","last_undegraded":"2024-08-16T19:08:59.545609+0000","last_fullsized":"2024-08-16T19:08:59.545609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455713+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:08:59.455713+0000","last_peered":"2024-08-16T19:08:59.455713+0000","last_clean":"2024-08-16T19:08:59.455713+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:08:59.455713+0000","last_undegraded":"2024-08-16T19:08:59.455713+0000","last_fullsized":"2024-08-16T19:08:59.455713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.466874+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:08:59.466874+0000","last_peered":"2024-08-16T19:08:59.466874+0000","last_clean":"2024-08-16T19:08:59.466874+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:08:59.466874+0000","last_undegraded":"2024-08-16T19:08:59.466874+0000","last_fullsized":"2024-08-16T19:08:59.466874+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456999+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:08:59.456999+0000","last_peered":"2024-08-16T19:08:59.456999+0000","last_clean":"2024-08-16T19:08:59.456999+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:08:59.456999+0000","last_undegraded":"2024-08-16T19:08:59.456999+0000","last_fullsized":"2024-08-16T19:08:59.456999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456539+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:08:59.456539+0000","last_peered":"2024-08-16T19:08:59.456539+0000","last_clean":"2024-08-16T19:08:59.456539+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:08:59.456539+0000","last_undegraded":"2024-08-16T19:08:59.456539+0000","last_fullsized":"2024-08-16T19:08:59.456539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462773+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:08:59.462773+0000","last_peered":"2024-08-16T19:08:59.462773+0000","last_clean":"2024-08-16T19:08:59.462773+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:08:59.462773+0000","last_undegraded":"2024-08-16T19:08:59.462773+0000","last_fullsized":"2024-08-16T19:08:59.462773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.545175+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:08:59.545175+0000","last_peered":"2024-08-16T19:08:59.545175+0000","last_clean":"2024-08-16T19:08:59.545175+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:08:59.545175+0000","last_undegraded":"2024-08-16T19:08:59.545175+0000","last_fullsized":"2024-08-16T19:08:59.545175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457436+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:08:59.457436+0000","last_peered":"2024-08-16T19:08:59.457436+0000","last_clean":"2024-08-16T19:08:59.457436+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:08:59.457436+0000","last_undegraded":"2024-08-16T19:08:59.457436+0000","last_fullsized":"2024-08-16T19:08:59.457436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457698+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:08:59.457698+0000","last_peered":"2024-08-16T19:08:59.457698+0000","last_clean":"2024-08-16T19:08:59.457698+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:08:59.457698+0000","last_undegraded":"2024-08-16T19:08:59.457698+0000","last_fullsized":"2024-08-16T19:08:59.457698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455841+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:08:59.455841+0000","last_peered":"2024-08-16T19:08:59.455841+0000","last_clean":"2024-08-16T19:08:59.455841+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:08:59.455841+0000","last_undegraded":"2024-08-16T19:08:59.455841+0000","last_fullsized":"2024-08-16T19:08:59.455841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457704+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:08:59.457704+0000","last_peered":"2024-08-16T19:08:59.457704+0000","last_clean":"2024-08-16T19:08:59.457704+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:08:59.457704+0000","last_undegraded":"2024-08-16T19:08:59.457704+0000","last_fullsized":"2024-08-16T19:08:59.457704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.455744+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:08:59.455744+0000","last_peered":"2024-08-16T19:08:59.455744+0000","last_clean":"2024-08-16T19:08:59.455744+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:08:59.455744+0000","last_undegraded":"2024-08-16T19:08:59.455744+0000","last_fullsized":"2024-08-16T19:08:59.455744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457744+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:08:59.457744+0000","last_peered":"2024-08-16T19:08:59.457744+0000","last_clean":"2024-08-16T19:08:59.457744+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:08:59.457744+0000","last_undegraded":"2024-08-16T19:08:59.457744+0000","last_fullsized":"2024-08-16T19:08:59.457744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.546015+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:08:59.546015+0000","last_peered":"2024-08-16T19:08:59.546015+0000","last_clean":"2024-08-16T19:08:59.546015+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:08:59.546015+0000","last_undegraded":"2024-08-16T19:08:59.546015+0000","last_fullsized":"2024-08-16T19:08:59.546015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.593295+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:08:59.593295+0000","last_peered":"2024-08-16T19:08:59.593295+0000","last_clean":"2024-08-16T19:08:59.593295+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:08:59.593295+0000","last_undegraded":"2024-08-16T19:08:59.593295+0000","last_fullsized":"2024-08-16T19:08:59.593295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.454659+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:08:59.454659+0000","last_peered":"2024-08-16T19:08:59.454659+0000","last_clean":"2024-08-16T19:08:59.454659+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:08:59.454659+0000","last_undegraded":"2024-08-16T19:08:59.454659+0000","last_fullsized":"2024-08-16T19:08:59.454659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.456837+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:08:59.456837+0000","last_peered":"2024-08-16T19:08:59.456837+0000","last_clean":"2024-08-16T19:08:59.456837+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:08:59.456837+0000","last_undegraded":"2024-08-16T19:08:59.456837+0000","last_fullsized":"2024-08-16T19:08:59.456837+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458460+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:08:59.458460+0000","last_peered":"2024-08-16T19:08:59.458460+0000","last_clean":"2024-08-16T19:08:59.458460+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:08:59.458460+0000","last_undegraded":"2024-08-16T19:08:59.458460+0000","last_fullsized":"2024-08-16T19:08:59.458460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457358+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:08:59.457358+0000","last_peered":"2024-08-16T19:08:59.457358+0000","last_clean":"2024-08-16T19:08:59.457358+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:08:59.457358+0000","last_undegraded":"2024-08-16T19:08:59.457358+0000","last_fullsized":"2024-08-16T19:08:59.457358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.458036+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:08:59.458036+0000","last_peered":"2024-08-16T19:08:59.458036+0000","last_clean":"2024-08-16T19:08:59.458036+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:08:59.458036+0000","last_undegraded":"2024-08-16T19:08:59.458036+0000","last_fullsized":"2024-08-16T19:08:59.458036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.457901+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:08:59.457901+0000","last_peered":"2024-08-16T19:08:59.457901+0000","last_clean":"2024-08-16T19:08:59.457901+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:08:59.457901+0000","last_undegraded":"2024-08-16T19:08:59.457901+0000","last_fullsized":"2024-08-16T19:08:59.457901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-08-16T19:08:59.462449+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:08:59.462449+0000","last_peered":"2024-08-16T19:08:59.462449+0000","last_clean":"2024-08-16T19:08:59.462449+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:08:59.462449+0000","last_undegraded":"2024-08-16T19:08:59.462449+0000","last_fullsized":"2024-08-16T19:08:59.462449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":537,"num_read_kb":349,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":208986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776164,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1724416,"data_stored":1157894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939689,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":512644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":508926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1105812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":504442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822715,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1100614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":508516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":503760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":567762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":567542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":505898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:09:41.921 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-16T19:09:41.921 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-16T19:09:41.921 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-16T19:09:41.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-16T19:09:42.185 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:42.618 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:42 smithi062 ceph-mon[32993]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:42.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 -- 172.21.15.62:0/4071108336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 msgr2=0x7f53e810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 --2- 172.21.15.62:0/4071108336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e810d770 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009ca0 tx=0x7f53e002a950 comp rx=0 tx=0).stop 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 -- 172.21.15.62:0/4071108336 shutdown_connections 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 --2- 172.21.15.62:0/4071108336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e810d770 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 --2- 172.21.15.62:0/4071108336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e81022a0 0x7f53e81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 --2- 172.21.15.62:0/4071108336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e8100b50 0x7f53e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 -- 172.21.15.62:0/4071108336 >> 172.21.15.62:0/4071108336 conn(0x7f53e80fc500 msgr2=0x7f53e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 -- 172.21.15.62:0/4071108336 shutdown_connections 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.697+0000 7f53f06a8640 1 -- 172.21.15.62:0/4071108336 wait complete. 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 Processor -- start 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 -- start start 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e8100b50 0x7f53e81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e81022a0 0x7f53e81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:42.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8110cb0 con 0x7f53e8102c20 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8110b30 con 0x7f53e8100b50 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53f06a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53e8110e30 con 0x7f53e81022a0 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53eec45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53eec45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42088/0 (socket says 172.21.15.62:42088) 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53eec45640 1 -- 172.21.15.62:0/1829865211 learned_addr learned my addr 172.21.15.62:0/1829865211 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53edc43640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e81022a0 0x7f53e81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.698+0000 7f53ee444640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e8100b50 0x7f53e81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 -- 172.21.15.62:0/1829865211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e81022a0 msgr2=0x7f53e81995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e81022a0 0x7f53e81995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 -- 172.21.15.62:0/1829865211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e8100b50 msgr2=0x7f53e81990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e8100b50 0x7f53e81990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 -- 172.21.15.62:0/1829865211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e81a4660 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53eec45640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f53e002ae70 tx=0x7f53e002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0034040 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0030920 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e81a4830 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0045480 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.699+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e81a4ce0 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.700+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e8100f30 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.701+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53e0037020 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.702+0000 7f53db7fe640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 0x7f53d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.702+0000 7f53ee444640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 0x7f53d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.702+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f53e00f0390 con 0x7f53e8102c20 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.703+0000 7f53ee444640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 0x7f53d40a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53e40058c0 tx=0x7f53e400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:42.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.706+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f53e0080df0 con 0x7f53e8102c20 2024-08-16T19:09:42.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:42 smithi155 ceph-mon[37413]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:42.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:42 smithi161 ceph-mon[37576]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:09:42.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.921+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f53e81a4f10 con 0x7f53e8102c20 2024-08-16T19:09:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.921+0000 7f53db7fe640 1 -- 172.21.15.62:0/1829865211 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f53e00bcf40 con 0x7f53e8102c20 2024-08-16T19:09:42.923 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:42.923 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-16T19:09:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 msgr2=0x7f53d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 0x7f53d40a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f53e40058c0 tx=0x7f53e400a250 comp rx=0 tx=0).stop 2024-08-16T19:09:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 msgr2=0x7f53e81a4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f53e002ae70 tx=0x7f53e002f5d0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 shutdown_connections 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53e8102c20 0x7f53e81a4120 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e81022a0 0x7f53e81995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f53d40a2c90 0x7f53d40a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 --2- 172.21.15.62:0/1829865211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e8100b50 0x7f53e81990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 >> 172.21.15.62:0/1829865211 conn(0x7f53e80fc500 msgr2=0x7f53e81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 shutdown_connections 2024-08-16T19:09:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:42.924+0000 7f53f06a8640 1 -- 172.21.15.62:0/1829865211 wait complete. 2024-08-16T19:09:43.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:43 smithi062 ceph-mon[32993]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:43.628 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:43 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1829865211' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-16T19:09:43.629 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-16T19:09:43.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:09:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:43 smithi155 ceph-mon[37413]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:43 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1829865211' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-16T19:09:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:43 smithi161 ceph-mon[37576]: pgmap v312: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:43 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1829865211' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-16T19:09:43.892 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 -- 172.21.15.62:0/550534763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 msgr2=0x7f18b810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 --2- 172.21.15.62:0/550534763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b810d660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009cd0 tx=0x7f18b002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 -- 172.21.15.62:0/550534763 shutdown_connections 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 --2- 172.21.15.62:0/550534763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b810d660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 --2- 172.21.15.62:0/550534763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 0x7f18b8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 --2- 172.21.15.62:0/550534763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b8106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 -- 172.21.15.62:0/550534763 >> 172.21.15.62:0/550534763 conn(0x7f18b80fc4b0 msgr2=0x7f18b80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 -- 172.21.15.62:0/550534763 shutdown_connections 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.335+0000 7f18bebaa640 1 -- 172.21.15.62:0/550534763 wait complete. 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 Processor -- start 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 -- start start 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 0x7f18b80fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b80fd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8110b90 con 0x7f18b8101130 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8110a10 con 0x7f18b8106790 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bebaa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8110d10 con 0x7f18b8100790 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:44.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18b7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42112/0 (socket says 172.21.15.62:42112) 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18b7fff640 1 -- 172.21.15.62:0/2524868171 learned_addr learned my addr 172.21.15.62:0/2524868171 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.336+0000 7f18bc946640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 0x7f18b80fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18bd147640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b80fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 -- 172.21.15.62:0/2524868171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 msgr2=0x7f18b80fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 0x7f18b80fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 -- 172.21.15.62:0/2524868171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 msgr2=0x7f18b80fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b80fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 -- 172.21.15.62:0/2524868171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81a65d0 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18bd147640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b80fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b7fff640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d770 tx=0x7f18a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4017070 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4005500 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a68c0 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4004e00 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.337+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a6e00 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.338+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8100bf0 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.339+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f18a4004fa0 con 0x7f18b8101130 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.340+0000 7f18b5ffb640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 0x7f18a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:44.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.340+0000 7f18bc946640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 0x7f18a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:44.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.340+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f18a40cef00 con 0x7f18b8101130 2024-08-16T19:09:44.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.341+0000 7f18bc946640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 0x7f18a80a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00bb60 tx=0x7f18ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:44.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.345+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f18a409b9a0 con 0x7f18b8101130 2024-08-16T19:09:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.557+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18b80fe0f0 con 0x7f18b8101130 2024-08-16T19:09:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.558+0000 7f18b5ffb640 1 -- 172.21.15.62:0/2524868171 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f18a409b2a0 con 0x7f18b8101130 2024-08-16T19:09:44.560 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:44.560 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:09:44.560 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 msgr2=0x7f18a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 0x7f18a80a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00bb60 tx=0x7f18ac00b390 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 msgr2=0x7f18b80fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d770 tx=0x7f18a400dc30 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 shutdown_connections 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18a80a2c90 0x7f18a80a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18b8106790 0x7f18b80fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b8101130 0x7f18b80fd100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 --2- 172.21.15.62:0/2524868171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18b8100790 0x7f18b80fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 >> 172.21.15.62:0/2524868171 conn(0x7f18b80fc4b0 msgr2=0x7f18b806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:44.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 shutdown_connections 2024-08-16T19:09:44.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:44.560+0000 7f18bebaa640 1 -- 172.21.15.62:0/2524868171 wait complete. 2024-08-16T19:09:45.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:09:45.399 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:45.820 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:45 smithi062 ceph-mon[32993]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:45.820 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:45 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2524868171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:45 smithi155 ceph-mon[37413]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:45 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2524868171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:45 smithi161 ceph-mon[37576]: pgmap v313: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:45 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2524868171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 -- 172.21.15.62:0/42700163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41018b0 msgr2=0x7f14a4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 --2- 172.21.15.62:0/42700163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41018b0 0x7f14a4101d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1494009930 tx=0x7f149402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 -- 172.21.15.62:0/42700163 shutdown_connections 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 --2- 172.21.15.62:0/42700163 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a4102250 0x7f14a410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 --2- 172.21.15.62:0/42700163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41018b0 0x7f14a4101d10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 --2- 172.21.15.62:0/42700163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a41078b0 0x7f14a4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.901+0000 7f14ac524640 1 -- 172.21.15.62:0/42700163 >> 172.21.15.62:0/42700163 conn(0x7f14a40fd5f0 msgr2=0x7f14a40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:45.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.902+0000 7f14ac524640 1 -- 172.21.15.62:0/42700163 shutdown_connections 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.902+0000 7f14ac524640 1 -- 172.21.15.62:0/42700163 wait complete. 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.902+0000 7f14ac524640 1 Processor -- start 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.902+0000 7f14ac524640 1 -- start start 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41018b0 0x7f14a41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a4102250 0x7f14a4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4111cb0 con 0x7f14a41078b0 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4111b30 con 0x7f14a4102250 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14ac524640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4111e30 con 0x7f14a41018b0 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42126/0 (socket says 172.21.15.62:42126) 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 -- 172.21.15.62:0/3480958343 learned_addr learned my addr 172.21.15.62:0/3480958343 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:45.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14a9abf640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a4102250 0x7f14a4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aa2c0640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41018b0 0x7f14a41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 -- 172.21.15.62:0/3480958343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41018b0 msgr2=0x7f14a41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41018b0 0x7f14a41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 -- 172.21.15.62:0/3480958343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a4102250 msgr2=0x7f14a4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a4102250 0x7f14a4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 -- 172.21.15.62:0/3480958343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a419d0a0 con 0x7f14a41078b0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.903+0000 7f14aaac1640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cce0 tx=0x7f149c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.904+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c00f450 con 0x7f14a41078b0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.904+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a419d390 con 0x7f14a41078b0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.904+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c00fad0 con 0x7f14a41078b0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.904+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c018b60 con 0x7f14a41078b0 2024-08-16T19:09:45.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.904+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41a4c70 con 0x7f14a41078b0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.905+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a4101d10 con 0x7f14a41078b0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.906+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f149c00fc80 con 0x7f14a41078b0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.907+0000 7f149b7fe640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 0x7f14900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.907+0000 7f14aa2c0640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 0x7f14900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.907+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f149c0cfd90 con 0x7f14a41078b0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.907+0000 7f14aa2c0640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 0x7f14900a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ab60 tx=0x7f14a000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:45.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:45.910+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f149c014030 con 0x7f14a41078b0 2024-08-16T19:09:46.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.124+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a41a4f10 con 0x7f14a41078b0 2024-08-16T19:09:46.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.125+0000 7f149b7fe640 1 -- 172.21.15.62:0/3480958343 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f149c09c830 con 0x7f14a41078b0 2024-08-16T19:09:46.126 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:46.126 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:09:46.127 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 msgr2=0x7f14900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 0x7f14900a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ab60 tx=0x7f14a000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 msgr2=0x7f14a419c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cce0 tx=0x7f149c00a620 comp rx=0 tx=0).stop 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 shutdown_connections 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a41078b0 0x7f14a419c9c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:46.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f14900a2c10 0x7f14900a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:46.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14a4102250 0x7f14a4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:46.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 --2- 172.21.15.62:0/3480958343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a41018b0 0x7f14a41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:46.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.127+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 >> 172.21.15.62:0/3480958343 conn(0x7f14a40fd5f0 msgr2=0x7f14a40fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:46.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.128+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 shutdown_connections 2024-08-16T19:09:46.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:46.128+0000 7f14ac524640 1 -- 172.21.15.62:0/3480958343 wait complete. 2024-08-16T19:09:46.784 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3480958343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:46.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:09:46.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3480958343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:46.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3480958343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 -- 172.21.15.62:0/674364407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c1067c0 msgr2=0x7f866c106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 --2- 172.21.15.62:0/674364407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c1067c0 0x7f866c106ba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8660009930 tx=0x7f866002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 -- 172.21.15.62:0/674364407 shutdown_connections 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 --2- 172.21.15.62:0/674364407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c101160 0x7f866c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 --2- 172.21.15.62:0/674364407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1007c0 0x7f866c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 --2- 172.21.15.62:0/674364407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c1067c0 0x7f866c106ba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 -- 172.21.15.62:0/674364407 >> 172.21.15.62:0/674364407 conn(0x7f866c0fc500 msgr2=0x7f866c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 -- 172.21.15.62:0/674364407 shutdown_connections 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.520+0000 7f86731f0640 1 -- 172.21.15.62:0/674364407 wait complete. 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 Processor -- start 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 -- start start 2024-08-16T19:09:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c1007c0 0x7f866c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1067c0 0x7f866c19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f866c110b80 con 0x7f866c101160 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f866c110a00 con 0x7f866c1007c0 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f86731f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f866c110d00 con 0x7f866c1067c0 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f866bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f866bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42146/0 (socket says 172.21.15.62:42146) 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.521+0000 7f866bfff640 1 -- 172.21.15.62:0/3113041724 learned_addr learned my addr 172.21.15.62:0/3113041724 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f867178d640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1067c0 0x7f866c19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f8670f8c640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c1007c0 0x7f866c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 -- 172.21.15.62:0/3113041724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1067c0 msgr2=0x7f866c19c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1067c0 0x7f866c19c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 -- 172.21.15.62:0/3113041724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c1007c0 msgr2=0x7f866c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c1007c0 0x7f866c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 -- 172.21.15.62:0/3113041724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c19d060 con 0x7f866c101160 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f866bfff640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f865800c920 tx=0x7f865800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658007c70 con 0x7f866c101160 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c19d350 con 0x7f866c101160 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8658007e10 con 0x7f866c101160 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658018890 con 0x7f866c101160 2024-08-16T19:09:47.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.522+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c1a4c30 con 0x7f866c101160 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.523+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866c100c20 con 0x7f866c101160 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.524+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8658018a70 con 0x7f866c101160 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.525+0000 7f8669ffb640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 0x7f865c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.525+0000 7f8670f8c640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 0x7f865c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.525+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f86580cf730 con 0x7f866c101160 2024-08-16T19:09:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.526+0000 7f8670f8c640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 0x7f865c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f8660031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.529+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8658014030 con 0x7f866c101160 2024-08-16T19:09:47.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.710+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f866c02d5e0 con 0x7f866c101160 2024-08-16T19:09:47.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.712+0000 7f8669ffb640 1 -- 172.21.15.62:0/3113041724 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24632 (secure 0 0 0) 0x7f865800de20 con 0x7f866c101160 2024-08-16T19:09:47.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:47.722 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:08:59.449917+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 msgr2=0x7f865c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 0x7f865c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f8660031490 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 msgr2=0x7f866c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f865800c920 tx=0x7f865800cde0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 shutdown_connections 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f866c1067c0 0x7f866c19c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f866c101160 0x7f866c1985f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f865c0a2c10 0x7f865c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 --2- 172.21.15.62:0/3113041724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f866c1007c0 0x7f866c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 >> 172.21.15.62:0/3113041724 conn(0x7f866c0fc500 msgr2=0x7f866c06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 shutdown_connections 2024-08-16T19:09:47.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:47.714+0000 7f86731f0640 1 -- 172.21.15.62:0/3113041724 wait complete. 2024-08-16T19:09:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:47 smithi155 ceph-mon[37413]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:47 smithi161 ceph-mon[37576]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:47 smithi062 ceph-mon[32993]: pgmap v314: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:48.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:09:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:48.691 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:48 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3113041724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:48.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:48 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3113041724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:48.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:48 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3113041724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:09:49.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2285031668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f741018e0 msgr2=0x7f7f74101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:49.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2285031668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f741018e0 0x7f7f74101d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009930 tx=0x7f7f6002a420 comp rx=0 tx=0).stop 2024-08-16T19:09:49.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2285031668 shutdown_connections 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2285031668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f74102280 0x7f7f7410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2285031668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f741018e0 0x7f7f74101d40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2285031668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741078e0 0x7f7f74107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.140+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2285031668 >> 172.21.15.62:0/2285031668 conn(0x7f7f740fd600 msgr2=0x7f7f740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2285031668 shutdown_connections 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2285031668 wait complete. 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 Processor -- start 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- start start 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741018e0 0x7f7f74074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 0x7f7f74074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74111d00 con 0x7f7f74102280 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74111b80 con 0x7f7f741078e0 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.141+0000 7f7f7ad16640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74111e80 con 0x7f7f741018e0 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42158/0 (socket says 172.21.15.62:42158) 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 -- 172.21.15.62:0/2986965861 learned_addr learned my addr 172.21.15.62:0/2986965861 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f78ab2640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741018e0 0x7f7f74074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f792b3640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 0x7f7f74074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 -- 172.21.15.62:0/2986965861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741018e0 msgr2=0x7f7f74074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741018e0 0x7f7f74074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 -- 172.21.15.62:0/2986965861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 msgr2=0x7f7f74074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 0x7f7f74074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 -- 172.21.15.62:0/2986965861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f741a6640 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f792b3640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 0x7f7f74074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f73fff640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009a50 tx=0x7f7f60004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6003f030 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f60003ad0 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f741a68d0 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.142+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60035070 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.143+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f741a6e00 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.144+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74101d40 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.145+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7f60042030 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.146+0000 7f7f71ffb640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 0x7f7f640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.146+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7f600f0060 con 0x7f7f74102280 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.146+0000 7f7f78ab2640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 0x7f7f640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.146+0000 7f7f78ab2640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 0x7f7f640a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800abe0 tx=0x7f7f6800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.152+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7f600bcb00 con 0x7f7f74102280 2024-08-16T19:09:49.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.364+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f741a7120 con 0x7f7f74102280 2024-08-16T19:09:49.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.365+0000 7f7f71ffb640 1 -- 172.21.15.62:0/2986965861 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f7f600bc400 con 0x7f7f74102280 2024-08-16T19:09:49.366 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:09:49.366 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:09:49.366 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 msgr2=0x7f7f640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 0x7f7f640a5150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800abe0 tx=0x7f7f6800a2d0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 msgr2=0x7f7f74074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009a50 tx=0x7f7f60004270 comp rx=0 tx=0).stop 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 shutdown_connections 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7f640a2c90 0x7f7f640a5150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f741078e0 0x7f7f74074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f74102280 0x7f7f74074760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 --2- 172.21.15.62:0/2986965861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f741018e0 0x7f7f74074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 >> 172.21.15.62:0/2986965861 conn(0x7f7f740fd600 msgr2=0x7f7f740fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.367+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 shutdown_connections 2024-08-16T19:09:49.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:49.368+0000 7f7f7ad16640 1 -- 172.21.15.62:0/2986965861 wait complete. 2024-08-16T19:09:49.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:49 smithi062 ceph-mon[32993]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:49.685 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:49 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2986965861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:49 smithi155 ceph-mon[37413]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:49 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2986965861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:49.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:49 smithi161 ceph-mon[37576]: pgmap v315: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:49.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:49 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2986965861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:09:50.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-16T19:08:05.356964+0000', 'modified': '2024-08-16T19:08:48.421714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24451, 'mds_3': 24407, 'mds_4': 24400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/659126409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 659126409}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 659126409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/3131957026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 3131957026}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 3131957026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.161:6833/1822368312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 1822368312}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 1822368312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.155:6833/2391473540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 2391473540}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 2391473540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/3063588307', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 3063588307}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 3063588307}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3346321531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3346321531}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3346321531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3410199866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3410199866}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3410199866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.161:6837/3769663917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3769663917}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3769663917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.161:6839/2650096635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2650096635}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2650096635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6839/2045382457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2045382457}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2045382457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24469, 24451, 24407, 24400]} 2024-08-16T19:09:50.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-16T19:09:50.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.l status 2024-08-16T19:09:50.297 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 -- 172.21.15.62:0/3132171567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9274101150 msgr2=0x7f927410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 --2- 172.21.15.62:0/3132171567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9274101150 0x7f927410d680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f926c009cd0 tx=0x7f926c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 -- 172.21.15.62:0/3132171567 shutdown_connections 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 --2- 172.21.15.62:0/3132171567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9274101150 0x7f927410d680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 --2- 172.21.15.62:0/3132171567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f9274100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 --2- 172.21.15.62:0/3132171567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92741067b0 0x7f9274106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 -- 172.21.15.62:0/3132171567 >> 172.21.15.62:0/3132171567 conn(0x7f92740fc4d0 msgr2=0x7f92740fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 -- 172.21.15.62:0/3132171567 shutdown_connections 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.756+0000 7f927b04c640 1 -- 172.21.15.62:0/3132171567 wait complete. 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 Processor -- start 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 -- start start 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f927419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 0x7f927419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f927410fb00 con 0x7f92741067b0 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f927410f980 con 0x7f92741007b0 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f927b04c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f927410fc80 con 0x7f9274101150 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42182/0 (socket says 172.21.15.62:42182) 2024-08-16T19:09:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 -- 172.21.15.62:0/3311304818 learned_addr learned my addr 172.21.15.62:0/3311304818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f9273fff640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 0x7f927419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f9278de8640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f927419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 -- 172.21.15.62:0/3311304818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 msgr2=0x7f927419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 0x7f927419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 -- 172.21.15.62:0/3311304818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 msgr2=0x7f927419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f927419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f92795e9640 1 -- 172.21.15.62:0/3311304818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927419f300 con 0x7f92741067b0 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f9278de8640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f927419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.757+0000 7f9273fff640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 0x7f927419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f92795e9640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f926c033040 tx=0x7f926c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f9271ffb640 1 -- 172.21.15.62:0/3311304818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004040 con 0x7f92741067b0 2024-08-16T19:09:50.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f9271ffb640 1 -- 172.21.15.62:0/3311304818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f926c0046c0 con 0x7f92741067b0 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927419f590 con 0x7f92741067b0 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f9271ffb640 1 -- 172.21.15.62:0/3311304818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c046480 con 0x7f92741067b0 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.758+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92741a6ed0 con 0x7f92741067b0 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.760+0000 7f9271ffb640 1 -- 172.21.15.62:0/3311304818 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f926c0305a0 con 0x7f92741067b0 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 Processor -- start 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 -- start start 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca2da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9271ffb640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 0x7f92640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca3ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56239c992710 con 0x56239ca2df50 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56239c92a770 con 0x56239ca2d630 2024-08-16T19:09:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f927d44a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239c98e6d0 con 0x56239ca38810 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9278de8640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 0x7f92640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9271ffb640 1 -- 172.21.15.62:0/3311304818 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f926c02e030 con 0x7f92741067b0 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca2da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca2da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:45800/0 (socket says 172.21.15.62:45800) 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 -- 172.21.15.62:0/2330293802 learned_addr learned my addr 172.21.15.62:0/2330293802 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f92795e9640 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca3ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 -- 172.21.15.62:0/2330293802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 msgr2=0x56239ca3ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca3ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 -- 172.21.15.62:0/2330293802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 msgr2=0x56239ca2da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca2da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f9273fff640 1 -- 172.21.15.62:0/2330293802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239c9313e0 con 0x56239ca2df50 2024-08-16T19:09:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.761+0000 7f92795e9640 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca3ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.762+0000 7f9278de8640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 0x7f92640a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9268005fd0 tx=0x7f926800b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.762+0000 7f9273fff640 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca382d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9260016840 tx=0x7f926000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4979191cb4dc29b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.762+0000 7f925effd640 1 -- 172.21.15.62:0/2330293802 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926001f070 con 0x56239ca2df50 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.762+0000 7f925effd640 1 -- 172.21.15.62:0/2330293802 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9260016dc0 con 0x56239ca2df50 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.762+0000 7f925effd640 1 -- 172.21.15.62:0/2330293802 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92600035f0 con 0x56239ca2df50 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- 172.21.15.62:0/2330293802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 msgr2=0x56239ca382d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca382d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9260016840 tx=0x7f926000d040 comp rx=0 tx=0).stop 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- 172.21.15.62:0/2330293802 shutdown_connections 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca3ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca382d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 --2- 172.21.15.62:0/2330293802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca2da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- 172.21.15.62:0/2330293802 >> 172.21.15.62:0/2330293802 conn(0x56239ca29bc0 msgr2=0x56239ca29fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- 172.21.15.62:0/2330293802 shutdown_connections 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- 172.21.15.62:0/2330293802 wait complete. 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 -- start start 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.763+0000 7f927d44a740 1 Processor -- start 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56239ca3b3e0 con 0x56239ca2df50 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56239c94d970 con 0x56239ca2d630 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239ca3b560 con 0x56239ca38810 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f927d44a740 10 client.0 _ll_register_callbacks cb 0x56239c6a17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42202/0 (socket says 172.21.15.62:42202) 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 -- 172.21.15.62:0/1150336203 learned_addr learned my addr 172.21.15.62:0/1150336203 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:50.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9278de8640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f92795e9640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 -- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 msgr2=0x56239ca361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 -- 172.21.15.62:0/1150336203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 msgr2=0x56239ca35750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca35750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 -- 172.21.15.62:0/1150336203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c7042b0 con 0x56239ca2df50 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f92795e9640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9278de8640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9273fff640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9260018b70 tx=0x7f9260018ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cd058eb91edeff server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9253fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.764+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260028060 con 0x56239ca2df50 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9260016dc0 con 0x56239ca2df50 2024-08-16T19:09:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92600186b0 con 0x56239ca2df50 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f926002f050 con 0x56239ca2df50 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 10 client.15048 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56239ca46fb0 con 0x56239ca2df50 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9260016dc0 con 0x56239ca2df50 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f9260022070 con 0x56239ca2df50 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 10 client.15048 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 20 client.15048 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239ca491b0 0x56239ca4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 4 client.15048 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.765+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239c99fc80 con 0x56239ca491b0 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.766+0000 7f9278de8640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239ca491b0 0x56239ca4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.766+0000 7f9278de8640 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239ca491b0 0x56239ca4a620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=507bf93ab4605390 server_cookie=6e3ee6590e0fda0e in_seq=0 out_seq=0 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.766+0000 7f9253fff640 10 client.15048 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f9253fff640 1 -- 172.21.15.62:0/1150336203 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56239c99fc80 con 0x56239ca491b0 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f9253fff640 10 client.15048 handle_command_reply: tid=0 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 client.15048 shutdown 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 msgr2=0x56239ca35c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9260018b70 tx=0x7f9260018ee0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 shutdown_connections 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239ca491b0 0x56239ca4a620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f9253fff640 0 client.15048 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca35c90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 --2- 172.21.15.62:0/1150336203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 >> 172.21.15.62:0/1150336203 conn(0x56239ca29bc0 msgr2=0x56239ca2f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 shutdown_connections 2024-08-16T19:09:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 1 -- 172.21.15.62:0/1150336203 wait complete. 2024-08-16T19:09:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.767+0000 7f927d44a740 20 client.15048 trim_cache size 0 max 16384 2024-08-16T19:09:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 Processor -- start 2024-08-16T19:09:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 -- start start 2024-08-16T19:09:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca43ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56239ca48960 con 0x56239ca2df50 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56239ca49030 con 0x56239ca2d630 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f927d44a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239ca49700 con 0x56239ca38810 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f9273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f9273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42204/0 (socket says 172.21.15.62:42204) 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.786+0000 7f9273fff640 1 -- 172.21.15.62:0/1025545770 learned_addr learned my addr 172.21.15.62:0/1025545770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9278de8640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f92795e9640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca43ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 -- 172.21.15.62:0/1025545770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 msgr2=0x56239ca43ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca43ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 -- 172.21.15.62:0/1025545770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 msgr2=0x56239ca49880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca49880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 -- 172.21.15.62:0/1025545770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239ca3b4f0 con 0x56239ca2df50 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f92795e9640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca43ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9278de8640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f9273fff640 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9260005700 tx=0x7f92600165b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5083e010c1f362c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f925e7fc640 1 -- 172.21.15.62:0/1025545770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260028070 con 0x56239ca2df50 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f925e7fc640 1 -- 172.21.15.62:0/1025545770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600186b0 con 0x56239ca2df50 2024-08-16T19:09:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.787+0000 7f925e7fc640 1 -- 172.21.15.62:0/1025545770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926001f040 con 0x56239ca2df50 2024-08-16T19:09:50.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- 172.21.15.62:0/1025545770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 msgr2=0x56239ca36020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9260005700 tx=0x7f92600165b0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- 172.21.15.62:0/1025545770 shutdown_connections 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca38810 0x56239ca43ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2df50 0x56239ca36020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 --2- 172.21.15.62:0/1025545770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca2d630 0x56239ca49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- 172.21.15.62:0/1025545770 >> 172.21.15.62:0/1025545770 conn(0x56239ca29bc0 msgr2=0x56239ca437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- 172.21.15.62:0/1025545770 shutdown_connections 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- 172.21.15.62:0/1025545770 wait complete. 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 -- start start 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.788+0000 7f927d44a740 1 Processor -- start 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca2df50 0x56239ca4c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 0x56239ca4c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56239ca4a4a0 con 0x56239ca2d630 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56239ca49dd0 con 0x56239ca38810 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56239ca4ab70 con 0x56239ca2df50 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f927d44a740 10 client.0 _ll_register_callbacks cb 0x7f926001d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42220/0 (socket says 172.21.15.62:42220) 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 -- 172.21.15.62:0/2504299967 learned_addr learned my addr 172.21.15.62:0/2504299967 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9273fff640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca2df50 0x56239ca4c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f92795e9640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 0x56239ca4c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 -- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca2df50 msgr2=0x56239ca4c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca2df50 0x56239ca4c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 -- 172.21.15.62:0/2504299967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 msgr2=0x56239ca4c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 0x56239ca4c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 -- 172.21.15.62:0/2504299967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239ca4d100 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f92795e9640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 0x56239ca4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9278de8640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f926800fe90 tx=0x7f92680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa22aa228b18a82f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.789+0000 7f9253fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268011640 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92680117e0 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92600186b0 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9268036050 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 10 client.15054 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56239cab38c0 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92600186b0 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f926801fc00 con 0x56239ca2d630 2024-08-16T19:09:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 10 client.15054 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 20 client.15054 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239cab8450 0x56239cab7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 4 client.15054 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.790+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "status"}) -- 0x56239c99a190 con 0x56239cab8450 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9273fff640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239cab8450 0x56239cab7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9253fff640 10 client.15054 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9273fff640 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239cab8450 0x56239cab7160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b0235446fe7ddd8 server_cookie=84cc9d1d178d9525 in_seq=0 out_seq=0 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9253fff640 1 -- 172.21.15.62:0/2504299967 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x56239c99a190 con 0x56239cab8450 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9253fff640 10 client.15054 handle_command_reply: tid=0 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 client.15054 shutdown 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 msgr2=0x56239ca53010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f926800fe90 tx=0x7f92680046c0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 shutdown_connections 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x56239cab8450 0x56239cab7160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56239ca38810 0x56239ca4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56239ca2df50 0x56239ca4c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 --2- 172.21.15.62:0/2504299967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56239ca2d630 0x56239ca53010 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 >> 172.21.15.62:0/2504299967 conn(0x56239ca29bc0 msgr2=0x56239ca28d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.791+0000 7f9253fff640 0 client.15054 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 shutdown_connections 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927d44a740 1 -- 172.21.15.62:0/2504299967 wait complete. 2024-08-16T19:09:50.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927d44a740 20 client.15054 trim_cache size 0 max 16384 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24400, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 63.380259705, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:09:50.798 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 147.619893201 2024-08-16T19:09:50.799 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:09:50.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 msgr2=0x7f92640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 0x7f92640a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9268005fd0 tx=0x7f926800b5d0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 msgr2=0x7f927419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f926c033040 tx=0x7f926c003970 comp rx=0 tx=0).stop 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 shutdown_connections 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92741067b0 0x7f927419ec20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9274101150 0x7f927419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92640a2c90 0x7f92640a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 --2- 172.21.15.62:0/3311304818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92741007b0 0x7f927419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.792+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 >> 172.21.15.62:0/3311304818 conn(0x7f92740fc4d0 msgr2=0x7f92740fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.793+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 shutdown_connections 2024-08-16T19:09:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:50.793+0000 7f927b04c640 1 -- 172.21.15.62:0/3311304818 wait complete. 2024-08-16T19:09:51.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.k status 2024-08-16T19:09:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:51 smithi155 ceph-mon[37413]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:51 smithi161 ceph-mon[37576]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:51.972 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:51 smithi062 ceph-mon[32993]: pgmap v316: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 -- 172.21.15.62:0/1134649385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 msgr2=0x7f9280106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 --2- 172.21.15.62:0/1134649385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f9280106b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f927c009930 tx=0x7f927c02a420 comp rx=0 tx=0).stop 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 -- 172.21.15.62:0/1134649385 shutdown_connections 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 --2- 172.21.15.62:0/1134649385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f928010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 --2- 172.21.15.62:0/1134649385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 0x7f9280100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 --2- 172.21.15.62:0/1134649385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f9280106b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.324+0000 7f92876db640 1 -- 172.21.15.62:0/1134649385 >> 172.21.15.62:0/1134649385 conn(0x7f92800fc4d0 msgr2=0x7f92800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- 172.21.15.62:0/1134649385 shutdown_connections 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- 172.21.15.62:0/1134649385 wait complete. 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 Processor -- start 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- start start 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 0x7f9280199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f92801995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9280110bb0 con 0x7f92801067b0 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9280110a30 con 0x7f9280101150 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.325+0000 7f92876db640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9280110d30 con 0x7f92801007b0 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42118/0 (socket says 172.21.15.62:42118) 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 -- 172.21.15.62:0/2858028869 learned_addr learned my addr 172.21.15.62:0/2858028869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:52.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285477640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 0x7f9280199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9284c76640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f92801995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 -- 172.21.15.62:0/2858028869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 msgr2=0x7f9280199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 0x7f9280199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 -- 172.21.15.62:0/2858028869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 msgr2=0x7f92801995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f92801995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 -- 172.21.15.62:0/2858028869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92801a4620 con 0x7f92801067b0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9284c76640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f92801995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f9285c78640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f927800eaa0 tx=0x7f927800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f92727fc640 1 -- 172.21.15.62:0/2858028869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927800cdb0 con 0x7f92801067b0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f92727fc640 1 -- 172.21.15.62:0/2858028869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9278005020 con 0x7f92801067b0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92801a47f0 con 0x7f92801067b0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f92727fc640 1 -- 172.21.15.62:0/2858028869 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780116b0 con 0x7f92801067b0 2024-08-16T19:09:52.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.326+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92801a4cb0 con 0x7f92801067b0 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.328+0000 7f92727fc640 1 -- 172.21.15.62:0/2858028869 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9278005560 con 0x7f92801067b0 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 Processor -- start 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 -- start start 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a557e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 0x55d3a5590010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3a54e3710 con 0x55d3a557ee50 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3a547b770 con 0x55d3a557e530 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9289ad9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3a54df6d0 con 0x55d3a558dc20 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f92727fc640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 0x7f926c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9285477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9285477640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42120/0 (socket says 172.21.15.62:42120) 2024-08-16T19:09:52.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.329+0000 7f9285477640 1 -- 172.21.15.62:0/2295146873 learned_addr learned my addr 172.21.15.62:0/2295146873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285c78640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 0x7f926c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9284c76640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 0x55d3a5590010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9284c76640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a557e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f92727fc640 1 -- 172.21.15.62:0/2858028869 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92780cf720 con 0x7f92801067b0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 -- 172.21.15.62:0/2295146873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 msgr2=0x55d3a5590010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 0x55d3a5590010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 -- 172.21.15.62:0/2295146873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 msgr2=0x55d3a557e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a557e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 -- 172.21.15.62:0/2295146873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a54823e0 con 0x55d3a557ee50 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9284c76640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 0x55d3a5590010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285c78640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 0x7f926c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9278098490 tx=0x7f9278051040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f9285477640 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f927c02a940 tx=0x7f927c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322ca4cd593d82ab server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f926b7fe640 1 -- 172.21.15.62:0/2295146873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c040070 con 0x55d3a557ee50 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f926b7fe640 1 -- 172.21.15.62:0/2295146873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c0319f0 con 0x55d3a557ee50 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.330+0000 7f926b7fe640 1 -- 172.21.15.62:0/2295146873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c035040 con 0x55d3a557ee50 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- 172.21.15.62:0/2295146873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 msgr2=0x55d3a558d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f927c02a940 tx=0x7f927c005c00 comp rx=0 tx=0).stop 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- 172.21.15.62:0/2295146873 shutdown_connections 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a558dc20 0x55d3a5590010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557ee50 0x55d3a558d6e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2295146873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a557e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- 172.21.15.62:0/2295146873 >> 172.21.15.62:0/2295146873 conn(0x55d3a557abc0 msgr2=0x55d3a557afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- 172.21.15.62:0/2295146873 shutdown_connections 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- 172.21.15.62:0/2295146873 wait complete. 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.331+0000 7f9289ad9740 1 -- start start 2024-08-16T19:09:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 Processor -- start 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a558afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3a55907f0 con 0x55d3a557e530 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3a549e970 con 0x55d3a558dc20 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3a5590970 con 0x55d3a557ee50 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9289ad9740 10 client.0 _ll_register_callbacks cb 0x55d3a51f27b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42130/0 (socket says 172.21.15.62:42130) 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 -- 172.21.15.62:0/1985254309 learned_addr learned my addr 172.21.15.62:0/1985254309 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9285477640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a558afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 -- 172.21.15.62:0/1985254309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 msgr2=0x55d3a558afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a558afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 -- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 msgr2=0x55d3a558b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9284c76640 1 -- 172.21.15.62:0/1985254309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a52552b0 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.332+0000 7f9285477640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a558afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9284c76640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9274013720 tx=0x7f9274013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1c439f80cb384 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274010d60 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92740113e0 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927c0319f0 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92740115e0 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9289ad9740 10 client.15066 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3a559c420 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c0319f0 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f927400f040 con 0x55d3a557e530 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.333+0000 7f9289ad9740 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:52.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9289ad9740 10 client.15066 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:09:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9289ad9740 20 client.15066 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a559ea20 0x55d3a559fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9289ad9740 4 client.15066 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:09:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a54f0c80 con 0x55d3a559ea20 2024-08-16T19:09:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9285477640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a559ea20 0x55d3a559fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9268ff9640 10 client.15066 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.334+0000 7f9285477640 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a559ea20 0x55d3a559fe90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c196c3941bb2402 server_cookie=5e7506b3304331e0 in_seq=0 out_seq=0 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.335+0000 7f9268ff9640 1 -- 172.21.15.62:0/1985254309 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3a54f0c80 con 0x55d3a559ea20 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.335+0000 7f9268ff9640 10 client.15066 handle_command_reply: tid=0 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.335+0000 7f9289ad9740 1 client.15066 shutdown 2024-08-16T19:09:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.335+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 msgr2=0x55d3a558aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9274013720 tx=0x7f9274013be0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 shutdown_connections 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a559ea20 0x55d3a559fe90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a558afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 --2- 172.21.15.62:0/1985254309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558aa80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 >> 172.21.15.62:0/1985254309 conn(0x55d3a557abc0 msgr2=0x55d3a5580920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9268ff9640 0 client.15066 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 shutdown_connections 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 1 -- 172.21.15.62:0/1985254309 wait complete. 2024-08-16T19:09:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.336+0000 7f9289ad9740 20 client.15066 trim_cache size 0 max 16384 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 Processor -- start 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 -- start start 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a559ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3a559e1d0 con 0x55d3a557e530 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3a559e8a0 con 0x55d3a558dc20 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9289ad9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3a559ef70 con 0x55d3a557ee50 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42144/0 (socket says 172.21.15.62:42144) 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 -- 172.21.15.62:0/2227293363 learned_addr learned my addr 172.21.15.62:0/2227293363 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9285477640 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a559ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 -- 172.21.15.62:0/2227293363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 msgr2=0x55d3a559ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a559ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 -- 172.21.15.62:0/2227293363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 msgr2=0x55d3a558a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.354+0000 7f9284c76640 1 -- 172.21.15.62:0/2227293363 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a557ec80 con 0x55d3a557e530 2024-08-16T19:09:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9284c76640 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9274000c00 tx=0x7f927400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b895f186174bdc server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f926affd640 1 -- 172.21.15.62:0/2227293363 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927402d020 con 0x55d3a557e530 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f926affd640 1 -- 172.21.15.62:0/2227293363 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9274017070 con 0x55d3a557e530 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f926affd640 1 -- 172.21.15.62:0/2227293363 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274029450 con 0x55d3a557e530 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 -- 172.21.15.62:0/2227293363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 msgr2=0x55d3a558d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9274000c00 tx=0x7f927400b490 comp rx=0 tx=0).stop 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 -- 172.21.15.62:0/2227293363 shutdown_connections 2024-08-16T19:09:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a558dc20 0x55d3a558a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a559ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2227293363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a557e530 0x55d3a558d1e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.355+0000 7f9289ad9740 1 -- 172.21.15.62:0/2227293363 >> 172.21.15.62:0/2227293363 conn(0x55d3a557abc0 msgr2=0x55d3a5599420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- 172.21.15.62:0/2227293363 shutdown_connections 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- 172.21.15.62:0/2227293363 wait complete. 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- start start 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 Processor -- start 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a55a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a55a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3a559fd10 con 0x55d3a558dc20 2024-08-16T19:09:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3a559f640 con 0x55d3a557e530 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3a55a03e0 con 0x55d3a557ee50 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.356+0000 7f9289ad9740 10 client.0 _ll_register_callbacks cb 0x7f927400bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9284c76640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42150/0 (socket says 172.21.15.62:42150) 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9284c76640 1 -- 172.21.15.62:0/2527887698 learned_addr learned my addr 172.21.15.62:0/2527887698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a55a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9284c76640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a55a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 -- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 msgr2=0x55d3a55a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a55a6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 -- 172.21.15.62:0/2527887698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 msgr2=0x55d3a55a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 -- 172.21.15.62:0/2527887698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a55a1c70 con 0x55d3a557ee50 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9284c76640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:52.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9285477640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a55a6550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f927c009900 tx=0x7f927c005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509dd8d8139ba39 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9268ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c054060 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9274017070 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.357+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f927c004240 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f927c05b050 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 10 client.24796 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d3a55a56f0 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9274017070 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f927c03c440 con 0x55d3a557ee50 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 10 client.24796 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 10 client.24796 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:09:52.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 20 client.24796 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a560d1b0 0x55d3a560bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 4 client.24796 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "status"}) -- 0x55d3a54eb190 con 0x55d3a560d1b0 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.358+0000 7f9284c76640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a560d1b0 0x55d3a560bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9268ff9640 10 client.24796 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:09:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9284c76640 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a560d1b0 0x55d3a560bec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3e23a64c34a5c7f server_cookie=d4d271b83509b3a0 in_seq=0 out_seq=0 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9268ff9640 1 -- 172.21.15.62:0/2527887698 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d3a54eb190 con 0x55d3a560d1b0 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9268ff9640 10 client.24796 handle_command_reply: tid=0 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 client.24796 shutdown 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 msgr2=0x55d3a55a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a55a6550 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f927c009900 tx=0x7f927c005e80 comp rx=0 tx=0).stop 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 shutdown_connections 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3a558dc20 0x55d3a55a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55d3a560d1b0 0x55d3a560bec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3a557ee50 0x55d3a55a6550 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 --2- 172.21.15.62:0/2527887698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3a557e530 0x55d3a55a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9268ff9640 0 client.24796 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.359+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 >> 172.21.15.62:0/2527887698 conn(0x55d3a557abc0 msgr2=0x55d3a5598c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 shutdown_connections 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f9289ad9740 1 -- 172.21.15.62:0/2527887698 wait complete. 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f9289ad9740 20 client.24796 trim_cache size 0 max 16384 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24407, 2024-08-16T19:09:52.365 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 65.954103489000005, 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 146.52694478399999 2024-08-16T19:09:52.366 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 msgr2=0x7f926c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 0x7f926c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9278098490 tx=0x7f9278051040 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 msgr2=0x7f92801a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f927800eaa0 tx=0x7f927800ef60 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 shutdown_connections 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92801067b0 0x7f92801a40e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f926c0a2c10 0x7f926c0a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9280101150 0x7f92801995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 --2- 172.21.15.62:0/2858028869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92801007b0 0x7f9280199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 >> 172.21.15.62:0/2858028869 conn(0x7f92800fc4d0 msgr2=0x7f92800fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:52.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 shutdown_connections 2024-08-16T19:09:52.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:52.360+0000 7f92876db640 1 -- 172.21.15.62:0/2858028869 wait complete. 2024-08-16T19:09:53.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-16T19:09:53.286 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:53.667 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:53 smithi062 ceph-mon[32993]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:53.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 -- 172.21.15.62:0/26518444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac103390 msgr2=0x7fb0ac10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 --2- 172.21.15.62:0/26518444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac103390 0x7fb0ac10b880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 -- 172.21.15.62:0/26518444 shutdown_connections 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 --2- 172.21.15.62:0/26518444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 --2- 172.21.15.62:0/26518444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac103390 0x7fb0ac10b880 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 --2- 172.21.15.62:0/26518444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac102a70 0x7fb0ac102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 -- 172.21.15.62:0/26518444 >> 172.21.15.62:0/26518444 conn(0x7fb0ac0fc4d0 msgr2=0x7fb0ac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 -- 172.21.15.62:0/26518444 shutdown_connections 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.749+0000 7fb0b3639640 1 -- 172.21.15.62:0/26518444 wait complete. 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 Processor -- start 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 -- start start 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac102a70 0x7fb0ac198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 0x7fb0ac198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac111ca0 con 0x7fb0ac10bea0 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac111b20 con 0x7fb0ac103390 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b3639640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac111e20 con 0x7fb0ac102a70 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42172/0 (socket says 172.21.15.62:42172) 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 -- 172.21.15.62:0/1813026901 learned_addr learned my addr 172.21.15.62:0/1813026901 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 0x7fb0ac198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac102a70 0x7fb0ac198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 -- 172.21.15.62:0/1813026901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac102a70 msgr2=0x7fb0ac198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac102a70 0x7fb0ac198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 -- 172.21.15.62:0/1813026901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 msgr2=0x7fb0ac198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 0x7fb0ac198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b1bd6640 1 -- 172.21.15.62:0/1813026901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac19d1c0 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.750+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 0x7fb0ac198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb0b1bd6640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c970 tx=0x7fb0a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb09e7fc640 1 -- 172.21.15.62:0/1813026901 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4007bb0 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb09e7fc640 1 -- 172.21.15.62:0/1813026901 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4007d50 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac19d4b0 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb09e7fc640 1 -- 172.21.15.62:0/1813026901 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400f450 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.751+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac1a4d90 con 0x7fb0ac10bea0 2024-08-16T19:09:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.753+0000 7fb09e7fc640 1 -- 172.21.15.62:0/1813026901 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0a4004f20 con 0x7fb0ac10bea0 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.753+0000 7fb0b5a37740 1 Processor -- start 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 -- start start 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b85f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b85f9e30 0x55d8b86086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8b855e710 con 0x55d8b8608c20 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8b84f6770 con 0x55d8b85f9510 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d8b855a6d0 con 0x55d8b85f9e30 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb09e7fc640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 0x7fb0980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42184/0 (socket says 172.21.15.62:42184) 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/1259439839 learned_addr learned my addr 172.21.15.62:0/1259439839 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b1bd6640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 0x7fb0980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b85f9e30 0x55d8b86086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb09e7fc640 1 -- 172.21.15.62:0/1813026901 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb0a40cf170 con 0x7fb0ac10bea0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b85f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/1259439839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b85f9e30 msgr2=0x55d8b86086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b85f9e30 0x55d8b86086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/1259439839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 msgr2=0x55d8b85f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b85f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.754+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/1259439839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b84fd3e0 con 0x55d8b8608c20 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0b1bd6640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 0x7fb0980a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097ee0 tx=0x7fb0a40909f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005ec0 tx=0x7fb0a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c49ba7167ba652a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0977fe640 1 -- 172.21.15.62:0/1259439839 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0035040 con 0x55d8b8608c20 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0977fe640 1 -- 172.21.15.62:0/1259439839 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a0005050 con 0x55d8b8608c20 2024-08-16T19:09:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0977fe640 1 -- 172.21.15.62:0/1259439839 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0004920 con 0x55d8b8608c20 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0b5a37740 1 -- 172.21.15.62:0/1259439839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 msgr2=0x55d8b860b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.755+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005ec0 tx=0x7fb0a0000bc0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- 172.21.15.62:0/1259439839 shutdown_connections 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b8608c20 0x55d8b860b010 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b85f9e30 0x55d8b86086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/1259439839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b85f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- 172.21.15.62:0/1259439839 >> 172.21.15.62:0/1259439839 conn(0x55d8b85f5bc0 msgr2=0x55d8b85f5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- 172.21.15.62:0/1259439839 shutdown_connections 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- 172.21.15.62:0/1259439839 wait complete. 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- start start 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 Processor -- start 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b86060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b86065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8b860b7f0 con 0x55d8b85f9510 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8b8519970 con 0x55d8b85f9e30 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.756+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d8b860b970 con 0x55d8b8608c20 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b5a37740 10 client.0 _ll_register_callbacks cb 0x55d8b826d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42198/0 (socket says 172.21.15.62:42198) 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2144855015 learned_addr learned my addr 172.21.15.62:0/2144855015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b86065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b86060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 msgr2=0x55d8b86065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b86065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2144855015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 msgr2=0x55d8b86060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b86060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2144855015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b82d02b0 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b86060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef57ccc11d2bba6e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0035070 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb0a0004ad0 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.757+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a0005050 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a0004cd0 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 10 client.15084 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8b8617480 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a0005050 con 0x55d8b85f9510 2024-08-16T19:09:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a003a040 con 0x55d8b85f9510 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 10 client.15084 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 20 client.15084 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b86187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 4 client.15084 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8b856bc80 con 0x55d8b8619570 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.758+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b86187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.759+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b86187d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9042fc13fa5d794d server_cookie=53bf5aefa0da343b in_seq=0 out_seq=0 2024-08-16T19:09:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.759+0000 7fb094ff9640 10 client.15084 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:09:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb094ff9640 1 -- 172.21.15.62:0/2144855015 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8b856bc80 con 0x55d8b8619570 2024-08-16T19:09:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb094ff9640 10 client.15084 handle_command_reply: tid=0 2024-08-16T19:09:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 client.15084 shutdown 2024-08-16T19:09:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 msgr2=0x55d8b8605b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a480 comp rx=0 tx=0).stop 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 shutdown_connections 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b86065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b86187d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb094ff9640 0 client.15084 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b86060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2144855015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b8605b60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 >> 172.21.15.62:0/2144855015 conn(0x55d8b85f5bc0 msgr2=0x55d8b85fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 shutdown_connections 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2144855015 wait complete. 2024-08-16T19:09:53.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.760+0000 7fb0b5a37740 20 client.15084 trim_cache size 0 max 16384 2024-08-16T19:09:53.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 Processor -- start 2024-08-16T19:09:53.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 -- start start 2024-08-16T19:09:53.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b8618ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b8603300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8b8618d20 con 0x55d8b85f9e30 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8b86193f0 con 0x55d8b85f9510 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d8b8619ac0 con 0x55d8b8608c20 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.778+0000 7fb0b13d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42206/0 (socket says 172.21.15.62:42206) 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 -- 172.21.15.62:0/2345553188 learned_addr learned my addr 172.21.15.62:0/2345553188 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b8618ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 -- 172.21.15.62:0/2345553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 msgr2=0x55d8b8603300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b8603300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 -- 172.21.15.62:0/2345553188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 msgr2=0x55d8b8618ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b8618ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 -- 172.21.15.62:0/2345553188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b85f9c60 con 0x55d8b85f9e30 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b8618ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005c40 tx=0x7fb0a8008610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9245631dc9fefad server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb096ffd640 1 -- 172.21.15.62:0/2345553188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8019330 con 0x55d8b85f9e30 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb096ffd640 1 -- 172.21.15.62:0/2345553188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a80199b0 con 0x55d8b85f9e30 2024-08-16T19:09:53.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.779+0000 7fb096ffd640 1 -- 172.21.15.62:0/2345553188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8010c10 con 0x55d8b85f9e30 2024-08-16T19:09:53.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2345553188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 msgr2=0x55d8b8602dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005c40 tx=0x7fb0a8008610 comp rx=0 tx=0).stop 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2345553188 shutdown_connections 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b8603300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9e30 0x55d8b8602dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2345553188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9510 0x55d8b8618ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2345553188 >> 172.21.15.62:0/2345553188 conn(0x55d8b85f5bc0 msgr2=0x55d8b8604430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2345553188 shutdown_connections 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2345553188 wait complete. 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 -- start start 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.780+0000 7fb0b5a37740 1 Processor -- start 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b861c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b861ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d8b861a860 con 0x55d8b85f9510 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d8b861a190 con 0x55d8b85f9e30 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d8b861af30 con 0x55d8b8608c20 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b5a37740 10 client.0 _ll_register_callbacks cb 0x7fb0a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42218/0 (socket says 172.21.15.62:42218) 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2099085739 learned_addr learned my addr 172.21.15.62:0/2099085739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b861c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 msgr2=0x55d8b861ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b861ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2099085739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 msgr2=0x55d8b861c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b861c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b0bd4640 1 -- 172.21.15.62:0/2099085739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b861d4e0 con 0x55d8b85f9510 2024-08-16T19:09:53.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.781+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b861c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b0bd4640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0a003c930 tx=0x7fb0a0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5beacaafb8d502 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0035070 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fb0a003ccb0 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a80199b0 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a0005050 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 10 client.15090 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d8b8683c80 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a80199b0 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a003a040 con 0x55d8b85f9510 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:09:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 20 client.15090 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b8687280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 4 client.15090 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.782+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "status"}) -- 0x55d8b8566190 con 0x55d8b8619570 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b8687280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb0b13d5640 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b8687280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f1de074c87f2264 server_cookie=a03889913ebfa873 in_seq=0 out_seq=0 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb094ff9640 10 client.15090 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb094ff9640 1 -- 172.21.15.62:0/2099085739 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d8b8566190 con 0x55d8b8619570 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb094ff9640 10 client.15090 handle_command_reply: tid=0 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.783+0000 7fb0b5a37740 1 client.15090 shutdown 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 msgr2=0x55d8b86233f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb0a003c930 tx=0x7fb0a0005db0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 shutdown_connections 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d8b8608c20 0x55d8b861ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d8b8619570 0x55d8b8687280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8b85f9e30 0x55d8b861c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 --2- 172.21.15.62:0/2099085739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d8b85f9510 0x55d8b86233f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb094ff9640 0 client.15090 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 >> 172.21.15.62:0/2099085739 conn(0x55d8b85f5bc0 msgr2=0x55d8b8613c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 shutdown_connections 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 1 -- 172.21.15.62:0/2099085739 wait complete. 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b5a37740 20 client.15090 trim_cache size 0 max 16384 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:09:53.789 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24451, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 68.046942731000001, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 133.49166078100001 2024-08-16T19:09:53.790 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:09:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 msgr2=0x7fb0980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 0x7fb0980a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097ee0 tx=0x7fb0a40909f0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 msgr2=0x7fb0ac19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400c970 tx=0x7fb0a400ce30 comp rx=0 tx=0).stop 2024-08-16T19:09:53.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 shutdown_connections 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0ac10bea0 0x7fb0ac19cae0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2c90 0x7fb0980a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ac103390 0x7fb0ac198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 --2- 172.21.15.62:0/1813026901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0ac102a70 0x7fb0ac198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.784+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 >> 172.21.15.62:0/1813026901 conn(0x7fb0ac0fc4d0 msgr2=0x7fb0ac0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.785+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 shutdown_connections 2024-08-16T19:09:53.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:53.785+0000 7fb0b3639640 1 -- 172.21.15.62:0/1813026901 wait complete. 2024-08-16T19:09:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:53 smithi155 ceph-mon[37413]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:53 smithi161 ceph-mon[37576]: pgmap v317: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:54.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-16T19:09:54.752 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.192+0000 7f92e2c63640 1 -- 172.21.15.62:0/3847808270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 msgr2=0x7f92dc100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.192+0000 7f92e2c63640 1 --2- 172.21.15.62:0/3847808270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc100c20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009930 tx=0x7f92c402a420 comp rx=0 tx=0).stop 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 -- 172.21.15.62:0/3847808270 shutdown_connections 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 --2- 172.21.15.62:0/3847808270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 0x7f92dc10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 --2- 172.21.15.62:0/3847808270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc100c20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 --2- 172.21.15.62:0/3847808270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 -- 172.21.15.62:0/3847808270 >> 172.21.15.62:0/3847808270 conn(0x7f92dc0fc500 msgr2=0x7f92dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 -- 172.21.15.62:0/3847808270 shutdown_connections 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 -- 172.21.15.62:0/3847808270 wait complete. 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 Processor -- start 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.193+0000 7f92e2c63640 1 -- start start 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 0x7f92dc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc111d60 con 0x7f92dc1007c0 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc111be0 con 0x7f92dc1067c0 2024-08-16T19:09:55.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e2c63640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc111ee0 con 0x7f92dc101160 2024-08-16T19:09:55.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42236/0 (socket says 172.21.15.62:42236) 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 -- 172.21.15.62:0/2714384062 learned_addr learned my addr 172.21.15.62:0/2714384062 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92dbfff640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 0x7f92dc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e1200640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 -- 172.21.15.62:0/2714384062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 msgr2=0x7f92dc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 0x7f92dc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 -- 172.21.15.62:0/2714384062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 msgr2=0x7f92dc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e09ff640 1 -- 172.21.15.62:0/2714384062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc19d2b0 con 0x7f92dc1007c0 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.194+0000 7f92e1200640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00ba50 tx=0x7f92cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92d9ffb640 1 -- 172.21.15.62:0/2714384062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc019070 con 0x7f92dc1007c0 2024-08-16T19:09:55.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92d9ffb640 1 -- 172.21.15.62:0/2714384062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92cc00cbe0 con 0x7f92dc1007c0 2024-08-16T19:09:55.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc19d5a0 con 0x7f92dc1007c0 2024-08-16T19:09:55.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92d9ffb640 1 -- 172.21.15.62:0/2714384062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0076e0 con 0x7f92dc1007c0 2024-08-16T19:09:55.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.195+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a4e80 con 0x7f92dc1007c0 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.197+0000 7f92e5061740 1 Processor -- start 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 -- start start 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d743a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d61d6a8710 con 0x55d61d743f50 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d61d640770 con 0x55d61d74e810 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e5061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d61d6a46d0 con 0x55d61d743630 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92d9ffb640 1 -- 172.21.15.62:0/2714384062 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f92cc01c070 con 0x7f92dc1007c0 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50156/0 (socket says 172.21.15.62:50156) 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e1200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d743a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92dbfff640 1 -- 172.21.15.62:0/3829787818 learned_addr learned my addr 172.21.15.62:0/3829787818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.198+0000 7f92e09ff640 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 -- 172.21.15.62:0/3829787818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 msgr2=0x55d61d743a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d743a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 -- 172.21.15.62:0/3829787818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 msgr2=0x55d61d74e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 -- 172.21.15.62:0/3829787818 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61d6473e0 con 0x55d61d74e810 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92dbfff640 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f92c402a940 tx=0x7f92c40316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111d119cd26ac197 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92d9ffb640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 0x7f92c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92d2ffd640 1 -- 172.21.15.62:0/3829787818 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4040070 con 0x55d61d74e810 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92e1200640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 0x7f92c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92d2ffd640 1 -- 172.21.15.62:0/3829787818 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c4005020 con 0x55d61d74e810 2024-08-16T19:09:55.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92d9ffb640 1 -- 172.21.15.62:0/2714384062 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92cc0ce9b0 con 0x7f92dc1007c0 2024-08-16T19:09:55.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.199+0000 7f92d2ffd640 1 -- 172.21.15.62:0/3829787818 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4035040 con 0x55d61d74e810 2024-08-16T19:09:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 -- 172.21.15.62:0/3829787818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 msgr2=0x55d61d750c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e1200640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 0x7f92c80a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92d400bad0 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f92c402a940 tx=0x7f92c40316e0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 -- 172.21.15.62:0/3829787818 shutdown_connections 2024-08-16T19:09:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d750c00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 --2- 172.21.15.62:0/3829787818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d743a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 -- 172.21.15.62:0/3829787818 >> 172.21.15.62:0/3829787818 conn(0x55d61d73fbc0 msgr2=0x55d61d73ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 -- 172.21.15.62:0/3829787818 shutdown_connections 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.200+0000 7f92e5061740 1 -- 172.21.15.62:0/3829787818 wait complete. 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 -- start start 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 Processor -- start 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743630 0x55d61d74b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 0x55d61d74c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d61d751430 con 0x55d61d743f50 2024-08-16T19:09:55.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d61d663970 con 0x55d61d743630 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d61d7515b0 con 0x55d61d74e810 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e5061740 10 client.0 _ll_register_callbacks cb 0x55d61d3b77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92dbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42254/0 (socket says 172.21.15.62:42254) 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92dbfff640 1 -- 172.21.15.62:0/4062367560 learned_addr learned my addr 172.21.15.62:0/4062367560 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e1200640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 0x55d61d74c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.201+0000 7f92e09ff640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743630 0x55d61d74b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 -- 172.21.15.62:0/4062367560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743630 msgr2=0x55d61d74b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743630 0x55d61d74b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 -- 172.21.15.62:0/4062367560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 msgr2=0x55d61d74bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 -- 172.21.15.62:0/4062367560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61d41a2b0 con 0x55d61d74e810 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92dbfff640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:09:55.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e1200640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 0x55d61d74c0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b8b0 tx=0x7f92d40047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfa1005007b6bea6 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d401c050 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92d4014860 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c4005020 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92d4014a60 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e5061740 10 client.24808 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.202+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d61d75d070 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c4005020 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f92d4006430 con 0x55d61d74e810 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 10 client.24808 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 10 client.24808 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 20 client.24808 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:55.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d75f7c0 0x55d61d760c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 4 client.24808 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d61d6b5c80 con 0x55d61d75f7c0 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e09ff640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d75f7c0 0x55d61d760c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92bbfff640 10 client.24808 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.203+0000 7f92e09ff640 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d75f7c0 0x55d61d760c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=443c75c529bf2689 server_cookie=5236b7b1c3d28a62 in_seq=0 out_seq=0 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92bbfff640 1 -- 172.21.15.62:0/4062367560 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d61d6b5c80 con 0x55d61d75f7c0 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92bbfff640 10 client.24808 handle_command_reply: tid=0 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92e5061740 1 client.24808 shutdown 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 msgr2=0x55d61d74c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.204+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 0x55d61d74c0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b8b0 tx=0x7f92d40047d0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 shutdown_connections 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d75f7c0 0x55d61d760c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d74e810 0x55d61d74c0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743f50 0x55d61d74bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 --2- 172.21.15.62:0/4062367560 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743630 0x55d61d74b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92bbfff640 0 client.24808 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 >> 172.21.15.62:0/4062367560 conn(0x55d61d73fbc0 msgr2=0x55d61d7458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 shutdown_connections 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 1 -- 172.21.15.62:0/4062367560 wait complete. 2024-08-16T19:09:55.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.205+0000 7f92e5061740 20 client.24808 trim_cache size 0 max 16384 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.222+0000 7f92e5061740 1 Processor -- start 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 -- start start 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d74ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743f50 0x55d61d760c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d61d75ef70 con 0x55d61d74e810 2024-08-16T19:09:55.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d61d75f640 con 0x55d61d743f50 2024-08-16T19:09:55.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e5061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d61d75fd10 con 0x55d61d743630 2024-08-16T19:09:55.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42264/0 (socket says 172.21.15.62:42264) 2024-08-16T19:09:55.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 -- 172.21.15.62:0/3826032048 learned_addr learned my addr 172.21.15.62:0/3826032048 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:55.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e09ff640 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d74ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92dbfff640 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743f50 0x55d61d760c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 -- 172.21.15.62:0/3826032048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 msgr2=0x55d61d74ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d74ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 -- 172.21.15.62:0/3826032048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743f50 msgr2=0x55d61d760c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743f50 0x55d61d760c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 -- 172.21.15.62:0/3826032048 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61d73f9f0 con 0x55d61d74e810 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.223+0000 7f92e1200640 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92d4008010 tx=0x7f92d40144b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51b37eba77e631f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92d27fc640 1 -- 172.21.15.62:0/3826032048 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d401c040 con 0x55d61d74e810 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92d27fc640 1 -- 172.21.15.62:0/3826032048 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4026070 con 0x55d61d74e810 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92d27fc640 1 -- 172.21.15.62:0/3826032048 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4033800 con 0x55d61d74e810 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 -- 172.21.15.62:0/3826032048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 msgr2=0x55d61d7488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92d4008010 tx=0x7f92d40144b0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 -- 172.21.15.62:0/3826032048 shutdown_connections 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d74e810 0x55d61d7488e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d743f50 0x55d61d760c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 --2- 172.21.15.62:0/3826032048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743630 0x55d61d74ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 -- 172.21.15.62:0/3826032048 >> 172.21.15.62:0/3826032048 conn(0x55d61d73fbc0 msgr2=0x55d61d7469b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 -- 172.21.15.62:0/3826032048 shutdown_connections 2024-08-16T19:09:55.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.224+0000 7f92e5061740 1 -- 172.21.15.62:0/3826032048 wait complete. 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 -- start start 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 Processor -- start 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743f50 0x55d61d7696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d747fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d61d760ab0 con 0x55d61d743630 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d61d7603e0 con 0x55d61d74e810 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d61d761180 con 0x55d61d743f50 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e5061740 10 client.0 _ll_register_callbacks cb 0x7f92d400b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e09ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e09ff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42274/0 (socket says 172.21.15.62:42274) 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.225+0000 7f92e09ff640 1 -- 172.21.15.62:0/2637958451 learned_addr learned my addr 172.21.15.62:0/2637958451 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92dbfff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743f50 0x55d61d7696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e1200640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d747fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 -- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743f50 msgr2=0x55d61d7696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743f50 0x55d61d7696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 -- 172.21.15.62:0/2637958451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 msgr2=0x55d61d747fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d747fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 -- 172.21.15.62:0/2637958451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61d73b130 con 0x55d61d743630 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e1200640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d747fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e09ff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92cc000f80 tx=0x7f92cc014d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1225aaba857b124 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0a3ad0 con 0x55d61d743630 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f92cc0cc250 con 0x55d61d743630 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d4026070 con 0x55d61d743630 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92cc0d2030 con 0x55d61d743630 2024-08-16T19:09:55.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e5061740 10 client.15108 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d61d7688a0 con 0x55d61d743630 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4026070 con 0x55d61d743630 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.226+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f92cc09a360 con 0x55d61d743630 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 20 client.15108 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d7ce470 0x55d61d7cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 4 client.15108 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "status"}) -- 0x55d61d6b0190 con 0x55d61d7ce470 2024-08-16T19:09:55.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.227+0000 7f92dbfff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d7ce470 0x55d61d7cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92dbfff640 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d7ce470 0x55d61d7cd180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6f7aaed20117860 server_cookie=c399aa94acb8a58d in_seq=0 out_seq=0 2024-08-16T19:09:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92bbfff640 10 client.15108 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:09:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92bbfff640 1 -- 172.21.15.62:0/2637958451 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d61d6b0190 con 0x55d61d7ce470 2024-08-16T19:09:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92bbfff640 10 client.15108 handle_command_reply: tid=0 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 client.15108 shutdown 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 msgr2=0x55d61d7691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92cc000f80 tx=0x7f92cc014d60 comp rx=0 tx=0).stop 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 shutdown_connections 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55d61d7ce470 0x55d61d7cd180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d61d74e810 0x55d61d747fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92bbfff640 0 client.15108 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d61d743f50 0x55d61d7696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 --2- 172.21.15.62:0/2637958451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d61d743630 0x55d61d7691b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 >> 172.21.15.62:0/2637958451 conn(0x55d61d73fbc0 msgr2=0x55d61d749f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 shutdown_connections 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 1 -- 172.21.15.62:0/2637958451 wait complete. 2024-08-16T19:09:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.228+0000 7f92e5061740 20 client.15108 trim_cache size 0 max 16384 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:09:55.234 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 70.502916730999999, 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 129.66362686299999 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 msgr2=0x7f92c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 0x7f92c80a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f92d400bad0 tx=0x7f92d4008040 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 msgr2=0x7f92dc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00ba50 tx=0x7f92cc00bf10 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 shutdown_connections 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f92c80a2c10 0x7f92c80a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92dc1067c0 0x7f92dc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92dc101160 0x7f92dc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 --2- 172.21.15.62:0/2714384062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92dc1007c0 0x7f92dc198300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 >> 172.21.15.62:0/2714384062 conn(0x7f92dc0fc500 msgr2=0x7f92dc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 shutdown_connections 2024-08-16T19:09:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:55.229+0000 7f92e2c63640 1 -- 172.21.15.62:0/2714384062 wait complete. 2024-08-16T19:09:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:55 smithi155 ceph-mon[37413]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:55.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:55 smithi161 ceph-mon[37576]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:55.891 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:55 smithi062 ceph-mon[32993]: pgmap v318: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-08-16T19:09:55.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-16T19:09:56.183 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:56.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 -- 172.21.15.62:0/3836424921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 msgr2=0x7f289810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 --2- 172.21.15.62:0/3836424921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f289810d680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2890009cd0 tx=0x7f289002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 -- 172.21.15.62:0/3836424921 shutdown_connections 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 --2- 172.21.15.62:0/3836424921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f289810d680 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 --2- 172.21.15.62:0/3836424921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 --2- 172.21.15.62:0/3836424921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f2898106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 -- 172.21.15.62:0/3836424921 >> 172.21.15.62:0/3836424921 conn(0x7f28980fc4d0 msgr2=0x7f28980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 -- 172.21.15.62:0/3836424921 shutdown_connections 2024-08-16T19:09:56.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.647+0000 7f289d3cc640 1 -- 172.21.15.62:0/3836424921 wait complete. 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 Processor -- start 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 -- start start 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f28981a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2898110bd0 con 0x7f2898101150 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2898110a50 con 0x7f28981067b0 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f289d3cc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2898110d50 con 0x7f28981007b0 2024-08-16T19:09:56.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42284/0 (socket says 172.21.15.62:42284) 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f28967fc640 1 -- 172.21.15.62:0/520644378 learned_addr learned my addr 172.21.15.62:0/520644378 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f28977fe640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f28981a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.648+0000 7f2896ffd640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 -- 172.21.15.62:0/520644378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 msgr2=0x7f2898199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 -- 172.21.15.62:0/520644378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 msgr2=0x7f28981a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f28981a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 -- 172.21.15.62:0/520644378 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981a45e0 con 0x7f2898101150 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28977fe640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f28981a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f2896ffd640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f28967fc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f288800b4a0 tx=0x7f288800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f288ffff640 1 -- 172.21.15.62:0/520644378 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888014070 con 0x7f2898101150 2024-08-16T19:09:56.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981a47b0 con 0x7f2898101150 2024-08-16T19:09:56.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f288ffff640 1 -- 172.21.15.62:0/520644378 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2888010af0 con 0x7f2898101150 2024-08-16T19:09:56.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f288ffff640 1 -- 172.21.15.62:0/520644378 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888017d10 con 0x7f2898101150 2024-08-16T19:09:56.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.649+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981a4cc0 con 0x7f2898101150 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.651+0000 7f288ffff640 1 -- 172.21.15.62:0/520644378 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2888017070 con 0x7f2898101150 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 Processor -- start 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 -- start start 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9612710 con 0x55c2c96ad630 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c2c95aa770 con 0x55c2c96adf50 2024-08-16T19:09:56.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f289f7ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2c960e6d0 con 0x55c2c96b8810 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f28977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f28977fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42286/0 (socket says 172.21.15.62:42286) 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f28977fe640 1 -- 172.21.15.62:0/321363686 learned_addr learned my addr 172.21.15.62:0/321363686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f288ffff640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 0x7f28840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f2896ffd640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.652+0000 7f28977fe640 1 -- 172.21.15.62:0/321363686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 msgr2=0x55c2c96bac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f288ffff640 1 -- 172.21.15.62:0/520644378 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f28880ce8a0 con 0x7f2898101150 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28967fc640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96bac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 -- 172.21.15.62:0/321363686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 msgr2=0x55c2c96b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 -- 172.21.15.62:0/321363686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c95b13e0 con 0x55c2c96ad630 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f2896ffd640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:56.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28967fc640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:09:56.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 0x7f28840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f28977fe640 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2890031650 tx=0x7f2890031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dd0e62e44421ee server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f288d7fa640 1 -- 172.21.15.62:0/321363686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890003e40 con 0x55c2c96ad630 2024-08-16T19:09:56.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f288d7fa640 1 -- 172.21.15.62:0/321363686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890036040 con 0x55c2c96ad630 2024-08-16T19:09:56.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.653+0000 7f288d7fa640 1 -- 172.21.15.62:0/321363686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890030920 con 0x55c2c96ad630 2024-08-16T19:09:56.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- 172.21.15.62:0/321363686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 msgr2=0x55c2c96ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2890031650 tx=0x7f2890031680 comp rx=0 tx=0).stop 2024-08-16T19:09:56.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f28977fe640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 0x7f28840a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f289004b000 tx=0x7f2890048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- 172.21.15.62:0/321363686 shutdown_connections 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 --2- 172.21.15.62:0/321363686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96ada10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- 172.21.15.62:0/321363686 >> 172.21.15.62:0/321363686 conn(0x55c2c96a9bc0 msgr2=0x55c2c96a9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- 172.21.15.62:0/321363686 shutdown_connections 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- 172.21.15.62:0/321363686 wait complete. 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.654+0000 7f289f7ca740 1 -- start start 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 Processor -- start 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96ad630 0x55c2c96b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96b8810 0x55c2c96b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96bf7f0 con 0x55c2c96adf50 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c2c95cd970 con 0x55c2c96b8810 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96bf970 con 0x55c2c96ad630 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f289f7ca740 10 client.0 _ll_register_callbacks cb 0x55c2c93217b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42294/0 (socket says 172.21.15.62:42294) 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 -- 172.21.15.62:0/3605505444 learned_addr learned my addr 172.21.15.62:0/3605505444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f2896ffd640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96ad630 0x55c2c96b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28977fe640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96b8810 0x55c2c96b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 -- 172.21.15.62:0/3605505444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96ad630 msgr2=0x55c2c96b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96ad630 0x55c2c96b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 -- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96b8810 msgr2=0x55c2c96b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.655+0000 7f28967fc640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96b8810 0x55c2c96b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f28967fc640 1 -- 172.21.15.62:0/3605505444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c93842b0 con 0x55c2c96adf50 2024-08-16T19:09:56.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f28967fc640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2888009420 tx=0x7f28880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6646fbd4e3643a14 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801e070 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f2888010ca0 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2890036040 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28880a39c0 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 10 client.15120 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2c96cb360 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890036040 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f2888062050 con 0x55c2c96adf50 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:09:56.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.656+0000 7f289f7ca740 20 client.15120 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c96cd950 0x55c2c96cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f289f7ca740 4 client.15120 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c961fc80 con 0x55c2c96cd950 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f2896ffd640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c96cd950 0x55c2c96cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f2896ffd640 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c96cd950 0x55c2c96cedc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fce6d77f3b0c106 server_cookie=634ac529f6832a3c in_seq=0 out_seq=0 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.657+0000 7f287e7fc640 10 client.15120 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f287e7fc640 1 -- 172.21.15.62:0/3605505444 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c961fc80 con 0x55c2c96cd950 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f287e7fc640 10 client.15120 handle_command_reply: tid=0 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 client.15120 shutdown 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 msgr2=0x55c2c96b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2888009420 tx=0x7f28880046c0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 shutdown_connections 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c96cd950 0x55c2c96cedc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96b8810 0x55c2c96b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96b5c30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 --2- 172.21.15.62:0/3605505444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96ad630 0x55c2c96b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 >> 172.21.15.62:0/3605505444 conn(0x55c2c96a9bc0 msgr2=0x55c2c96a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f287e7fc640 0 client.15120 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 shutdown_connections 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 1 -- 172.21.15.62:0/3605505444 wait complete. 2024-08-16T19:09:56.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.658+0000 7f289f7ca740 20 client.15120 trim_cache size 0 max 16384 2024-08-16T19:09:56.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.676+0000 7f289f7ca740 1 Processor -- start 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.676+0000 7f289f7ca740 1 -- start start 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96cd100 con 0x55c2c96ad630 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96cd7d0 con 0x55c2c96adf50 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f289f7ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96cdea0 con 0x55c2c96b8810 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42308/0 (socket says 172.21.15.62:42308) 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 -- 172.21.15.62:0/763401442 learned_addr learned my addr 172.21.15.62:0/763401442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f28967fc640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f28977fe640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 -- 172.21.15.62:0/763401442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 msgr2=0x55c2c96a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 -- 172.21.15.62:0/763401442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 msgr2=0x55c2c96b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 -- 172.21.15.62:0/763401442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c96bf900 con 0x55c2c96ad630 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f28967fc640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:56.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.677+0000 7f2896ffd640 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f288000e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc47dc3dd359d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f288cff9640 1 -- 172.21.15.62:0/763401442 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880005650 con 0x55c2c96ad630 2024-08-16T19:09:56.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f288cff9640 1 -- 172.21.15.62:0/763401442 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28800035f0 con 0x55c2c96ad630 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f288cff9640 1 -- 172.21.15.62:0/763401442 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880004e60 con 0x55c2c96ad630 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 -- 172.21.15.62:0/763401442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 msgr2=0x55c2c96cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f288000e490 comp rx=0 tx=0).stop 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 -- 172.21.15.62:0/763401442 shutdown_connections 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96adf50 0x55c2c96b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 --2- 172.21.15.62:0/763401442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96ad630 0x55c2c96cd950 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 -- 172.21.15.62:0/763401442 >> 172.21.15.62:0/763401442 conn(0x55c2c96a9bc0 msgr2=0x55c2c96b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 -- 172.21.15.62:0/763401442 shutdown_connections 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.678+0000 7f289f7ca740 1 -- 172.21.15.62:0/763401442 wait complete. 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 -- start start 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 Processor -- start 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96ad630 0x55c2c96d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96cec40 con 0x55c2c96adf50 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96ce570 con 0x55c2c96ad630 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2c96cf310 con 0x55c2c96b8810 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f289f7ca740 10 client.0 _ll_register_callbacks cb 0x7f2888017f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f28967fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42320/0 (socket says 172.21.15.62:42320) 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f28967fc640 1 -- 172.21.15.62:0/75918548 learned_addr learned my addr 172.21.15.62:0/75918548 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.679+0000 7f2896ffd640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96ad630 0x55c2c96d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 -- 172.21.15.62:0/75918548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 msgr2=0x55c2c96d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 -- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96ad630 msgr2=0x55c2c96d7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96ad630 0x55c2c96d7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 -- 172.21.15.62:0/75918548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c96d1170 con 0x55c2c96adf50 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f28967fc640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2888099000 tx=0x7f2888009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800a227d32963e03 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801e040 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f28880b9440 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28800035f0 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f288809a890 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f289f7ca740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c2c96d5bd0 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28800035f0 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f2888062070 con 0x55c2c96adf50 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.680+0000 7f289f7ca740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f289f7ca740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f289f7ca740 20 client.15126 populate_metadata read hostname 'smithi062' 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c973c7a0 0x55c2c973b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f289f7ca740 4 client.15126 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "status"}) -- 0x55c2c961a190 con 0x55c2c973c7a0 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f2896ffd640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c973c7a0 0x55c2c973b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f287e7fc640 10 client.15126 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f2896ffd640 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c973c7a0 0x55c2c973b4b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a38c1f93fbfaf8 server_cookie=54b117929a422551 in_seq=0 out_seq=0 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f287e7fc640 1 -- 172.21.15.62:0/75918548 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c2c961a190 con 0x55c2c973c7a0 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.681+0000 7f287e7fc640 10 client.15126 handle_command_reply: tid=0 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 client.15126 shutdown 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 msgr2=0x55c2c96d7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2888099000 tx=0x7f2888009390 comp rx=0 tx=0).stop 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 shutdown_connections 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2c973c7a0 0x55c2c973b4b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c2c96b8810 0x55c2c96d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f287e7fc640 0 client.15126 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:09:56.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2c96adf50 0x55c2c96d7980 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 --2- 172.21.15.62:0/75918548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2c96ad630 0x55c2c96d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 >> 172.21.15.62:0/75918548 conn(0x55c2c96a9bc0 msgr2=0x55c2c96b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 shutdown_connections 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 1 -- 172.21.15.62:0/75918548 wait complete. 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289f7ca740 20 client.15126 trim_cache size 0 max 16384 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24476, 2024-08-16T19:09:56.687 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 111.104427899, 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 90, 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 122.14260043500001 2024-08-16T19:09:56.688 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 msgr2=0x7f28840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 0x7f28840a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f289004b000 tx=0x7f2890048000 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 msgr2=0x7f2898199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f288800b4a0 tx=0x7f288800b960 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 shutdown_connections 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.682+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f28840a2c10 0x7f28840a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28981067b0 0x7f28981a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2898101150 0x7f2898199560 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 --2- 172.21.15.62:0/520644378 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28981007b0 0x7f2898199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 >> 172.21.15.62:0/520644378 conn(0x7f28980fc4d0 msgr2=0x7f28980fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 shutdown_connections 2024-08-16T19:09:56.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:56.683+0000 7f289d3cc640 1 -- 172.21.15.62:0/520644378 wait complete. 2024-08-16T19:09:57.442 INFO:tasks.cephadm:Setup complete, yielding 2024-08-16T19:09:57.442 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-16T19:09:57.452 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:09:57.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-16T19:09:57.743 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:57.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:57 smithi062 ceph-mon[32993]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:57 smithi155 ceph-mon[37413]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:57 smithi161 ceph-mon[37576]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-08-16T19:09:58.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- 172.21.15.62:0/3395745563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c1034f0 msgr2=0x7f7a7c109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:58.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 --2- 172.21.15.62:0/3395745563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c109d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009ca0 tx=0x7f7a7402a8c0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- 172.21.15.62:0/3395745563 shutdown_connections 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 --2- 172.21.15.62:0/3395745563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c109d80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 --2- 172.21.15.62:0/3395745563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c102b30 0x7f7a7c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 --2- 172.21.15.62:0/3395745563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 0x7f7a7c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- 172.21.15.62:0/3395745563 >> 172.21.15.62:0/3395745563 conn(0x7f7a7c0fd0c0 msgr2=0x7f7a7c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- 172.21.15.62:0/3395745563 shutdown_connections 2024-08-16T19:09:58.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- 172.21.15.62:0/3395745563 wait complete. 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 Processor -- start 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.222+0000 7f7a8294d640 1 -- start start 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 0x7f7a7c075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d600 con 0x7f7a7c102b30 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d480 con 0x7f7a7c1034f0 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a8294d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c10d780 con 0x7f7a7c101930 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:58.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42342/0 (socket says 172.21.15.62:42342) 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 -- 172.21.15.62:0/1908876043 learned_addr learned my addr 172.21.15.62:0/1908876043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7bfff640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 0x7f7a7c075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a80eea640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 -- 172.21.15.62:0/1908876043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 msgr2=0x7f7a7c075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 0x7f7a7c075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 -- 172.21.15.62:0/1908876043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 msgr2=0x7f7a7c074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a7b7fe640 1 -- 172.21.15.62:0/1908876043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c075110 con 0x7f7a7c102b30 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.223+0000 7f7a80eea640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a7b7fe640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d950 tx=0x7f7a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a70017030 con 0x7f7a7c102b30 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1a1d50 con 0x7f7a7c102b30 2024-08-16T19:09:58.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a70004020 con 0x7f7a7c102b30 2024-08-16T19:09:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a700047a0 con 0x7f7a7c102b30 2024-08-16T19:09:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.224+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a2250 con 0x7f7a7c102b30 2024-08-16T19:09:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.225+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c108c60 con 0x7f7a7c102b30 2024-08-16T19:09:58.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.226+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a700041d0 con 0x7f7a7c102b30 2024-08-16T19:09:58.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.227+0000 7f7a797fa640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:58.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.227+0000 7f7a7bfff640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:58.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.227+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7f7a700cf4e0 con 0x7f7a7c102b30 2024-08-16T19:09:58.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.227+0000 7f7a7bfff640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c005fd0 tx=0x7f7a6c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:58.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.230+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7a7009c000 con 0x7f7a7c102b30 2024-08-16T19:09:58.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.423+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7a7c062360 con 0x7f7a640a2c90 2024-08-16T19:09:58.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.424+0000 7f7a797fa640 1 -- 172.21.15.62:0/1908876043 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7a7c062360 con 0x7f7a640a2c90 2024-08-16T19:09:58.425 INFO:teuthology.orchestra.run.smithi062.stdout:Backend: cephadm 2024-08-16T19:09:58.425 INFO:teuthology.orchestra.run.smithi062.stdout:Available: Yes 2024-08-16T19:09:58.425 INFO:teuthology.orchestra.run.smithi062.stdout:Paused: No 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.426+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 msgr2=0x7f7a640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.426+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c005fd0 tx=0x7f7a6c00b390 comp rx=0 tx=0).stop 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.426+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 msgr2=0x7f7a7c074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.426+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000d950 tx=0x7f7a7000de10 comp rx=0 tx=0).stop 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 shutdown_connections 2024-08-16T19:09:58.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a7c1034f0 0x7f7a7c074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a7c102b30 0x7f7a7c074310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 --2- 172.21.15.62:0/1908876043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a7c101930 0x7f7a7c075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 >> 172.21.15.62:0/1908876043 conn(0x7f7a7c0fd0c0 msgr2=0x7f7a7c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 shutdown_connections 2024-08-16T19:09:58.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:58.427+0000 7f7a8294d640 1 -- 172.21.15.62:0/1908876043 wait complete. 2024-08-16T19:09:58.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-16T19:09:58.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-08-16T19:09:58.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:58 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-08-16T19:09:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-16T19:09:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-08-16T19:09:58.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:58 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-08-16T19:09:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-16T19:09:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-08-16T19:09:58.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:58 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-08-16T19:09:59.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-16T19:09:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: osdmap e91: 12 total, 12 up, 12 in 2024-08-16T19:09:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:09:59 smithi155 ceph-mon[37413]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: osdmap e91: 12 total, 12 up, 12 in 2024-08-16T19:09:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:09:59 smithi161 ceph-mon[37576]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-16T19:09:59.898 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: osdmap e91: 12 total, 12 up, 12 in 2024-08-16T19:09:59.899 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:09:59 smithi062 ceph-mon[32993]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-16T19:09:59.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- 172.21.15.62:0/933333210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 msgr2=0x7f4cb4102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:59.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 --2- 172.21.15.62:0/933333210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4102fb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009900 tx=0x7f4ca802a410 comp rx=0 tx=0).stop 2024-08-16T19:09:59.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- 172.21.15.62:0/933333210 shutdown_connections 2024-08-16T19:09:59.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 --2- 172.21.15.62:0/933333210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 0x7f4cb4109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 --2- 172.21.15.62:0/933333210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4102fb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 --2- 172.21.15.62:0/933333210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 0x7f4cb4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- 172.21.15.62:0/933333210 >> 172.21.15.62:0/933333210 conn(0x7f4cb40fd0c0 msgr2=0x7f4cb40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- 172.21.15.62:0/933333210 shutdown_connections 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- 172.21.15.62:0/933333210 wait complete. 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 Processor -- start 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.981+0000 7f4cb9844640 1 -- start start 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 0x7f4cb4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 0x7f4cb419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410d590 con 0x7f4cb4102b30 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410d410 con 0x7f4cb41034f0 2024-08-16T19:09:59.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb9844640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410d710 con 0x7f4cb4101930 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42358/0 (socket says 172.21.15.62:42358) 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 -- 172.21.15.62:0/218547878 learned_addr learned my addr 172.21.15.62:0/218547878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb2ffd640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 0x7f4cb4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb37fe640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 0x7f4cb419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 -- 172.21.15.62:0/218547878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 msgr2=0x7f4cb4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 0x7f4cb4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 -- 172.21.15.62:0/218547878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 msgr2=0x7f4cb419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 0x7f4cb419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 -- 172.21.15.62:0/218547878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb419f800 con 0x7f4cb4102b30 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.982+0000 7f4cb27fc640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009230 tx=0x7f4ca8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.983+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca803f030 con 0x7f4cb4102b30 2024-08-16T19:09:59.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.983+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca8003ad0 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.983+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb419fa90 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.983+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8035070 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.983+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb419ffc0 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.984+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb4108cd0 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.985+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4ca8042030 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.985+0000 7f4cb8842640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.986+0000 7f4cb2ffd640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.986+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f4ca80ef900 con 0x7f4cb4102b30 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.986+0000 7f4cb2ffd640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4102990 tx=0x7f4ca4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:09:59.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:09:59.989+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4ca80bc4b0 con 0x7f4cb4102b30 2024-08-16T19:10:00.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.177+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4cb4062360 con 0x7f4c9c0a2c30 2024-08-16T19:10:00.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.190+0000 7f4cb8842640 1 -- 172.21.15.62:0/218547878 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f4cb4062360 con 0x7f4c9c0a2c30 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.a smithi062 running (2m) 57s ago 2m 15.6M - 19.1.0-1363-gd863d21c ba08250ef864 cb9791b7d2cb 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.b smithi155 running (2m) 58s ago 2m 26.7M - 19.1.0-1363-gd863d21c ba08250ef864 5a154aa5c529 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.c smithi161 running (2m) 58s ago 2m 17.5M - 19.1.0-1363-gd863d21c ba08250ef864 45dc381d5103 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.d smithi062 running (2m) 57s ago 2m 14.4M - 19.1.0-1363-gd863d21c ba08250ef864 0a98a8064d1a 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.e smithi155 running (2m) 58s ago 2m 15.7M - 19.1.0-1363-gd863d21c ba08250ef864 fd8951e1e8d8 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.f smithi161 running (2m) 58s ago 2m 14.9M - 19.1.0-1363-gd863d21c ba08250ef864 98f2754eb4de 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.g smithi062 running (2m) 57s ago 2m 15.6M - 19.1.0-1363-gd863d21c ba08250ef864 5db14e43d4f3 2024-08-16T19:10:00.191 INFO:teuthology.orchestra.run.smithi062.stdout:mds.h smithi155 running (2m) 58s ago 2m 18.8M - 19.1.0-1363-gd863d21c ba08250ef864 cdc114c91707 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mds.i smithi161 running (2m) 58s ago 2m 17.5M - 19.1.0-1363-gd863d21c ba08250ef864 e854ba103aeb 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mds.j smithi062 running (2m) 57s ago 2m 13.4M - 19.1.0-1363-gd863d21c ba08250ef864 72c5dbc0bdaf 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mds.k smithi155 running (2m) 58s ago 2m 19.7M - 19.1.0-1363-gd863d21c ba08250ef864 3911eb8823cd 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mds.l smithi161 running (2m) 58s ago 2m 18.8M - 19.1.0-1363-gd863d21c ba08250ef864 0dc5272ab413 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.x smithi062 *:9283,8765 running (8m) 57s ago 8m 506M - 19.1.0-1363-gd863d21c ba08250ef864 0d3d747f003f 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.y smithi155 *:8443,8765 running (7m) 58s ago 7m 443M - 19.1.0-1363-gd863d21c ba08250ef864 c4ffd3fc9417 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mgr.z smithi161 *:8443,8765 running (7m) 58s ago 7m 442M - 19.1.0-1363-gd863d21c ba08250ef864 85dbe540ff06 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mon.a smithi062 running (8m) 57s ago 8m 51.5M 2048M 19.1.0-1363-gd863d21c ba08250ef864 aef3f972923c 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mon.b smithi155 running (7m) 58s ago 7m 41.4M 2048M 19.1.0-1363-gd863d21c ba08250ef864 8afd90ef9138 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:mon.c smithi161 running (7m) 58s ago 7m 41.4M 2048M 19.1.0-1363-gd863d21c ba08250ef864 171681b2b1a4 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 smithi062 running (6m) 57s ago 6m 46.7M 4272M 19.1.0-1363-gd863d21c ba08250ef864 a76aa2dc46aa 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 smithi155 running (6m) 58s ago 6m 46.5M 4272M 19.1.0-1363-gd863d21c ba08250ef864 2a209a236974 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 smithi161 running (6m) 58s ago 6m 55.2M 4272M 19.1.0-1363-gd863d21c ba08250ef864 7f674a498cc0 2024-08-16T19:10:00.192 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 smithi062 running (5m) 57s ago 5m 44.7M 4272M 19.1.0-1363-gd863d21c ba08250ef864 df3808467e7e 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 smithi155 running (5m) 58s ago 5m 54.1M 4272M 19.1.0-1363-gd863d21c ba08250ef864 cfefa2060a92 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 smithi161 running (5m) 58s ago 5m 47.4M 4272M 19.1.0-1363-gd863d21c ba08250ef864 de4944b9a03e 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 smithi062 running (4m) 57s ago 4m 52.9M 4272M 19.1.0-1363-gd863d21c ba08250ef864 101ea39e160b 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 smithi155 running (4m) 58s ago 4m 47.2M 4272M 19.1.0-1363-gd863d21c ba08250ef864 1b71f96fc906 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 smithi161 running (4m) 58s ago 4m 52.1M 4272M 19.1.0-1363-gd863d21c ba08250ef864 174b8e397ec4 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 smithi062 running (3m) 57s ago 3m 50.9M 4272M 19.1.0-1363-gd863d21c ba08250ef864 18984ba59aa7 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 smithi155 running (3m) 58s ago 3m 43.4M 4272M 19.1.0-1363-gd863d21c ba08250ef864 11b160c9751c 2024-08-16T19:10:00.193 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 smithi161 running (2m) 58s ago 2m 44.7M 4272M 19.1.0-1363-gd863d21c ba08250ef864 00d2b5dc6bc7 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.192+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 msgr2=0x7f4c9c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.192+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4102990 tx=0x7f4ca4005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.192+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 msgr2=0x7f4cb4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.192+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009230 tx=0x7f4ca8031b60 comp rx=0 tx=0).stop 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 shutdown_connections 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f4c9c0a2c30 0x7f4c9c0a50f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cb41034f0 0x7f4cb419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:00.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cb4102b30 0x7f4cb4198110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:00.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 --2- 172.21.15.62:0/218547878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cb4101930 0x7f4cb4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:00.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 >> 172.21.15.62:0/218547878 conn(0x7f4cb40fd0c0 msgr2=0x7f4cb4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:00.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 shutdown_connections 2024-08-16T19:10:00.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:00.193+0000 7f4cb9844640 1 -- 172.21.15.62:0/218547878 wait complete. 2024-08-16T19:10:00.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:00 smithi062 ceph-mon[32993]: osdmap e92: 12 total, 12 up, 12 in 2024-08-16T19:10:00.838 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:00 smithi062 ceph-mon[32993]: overall HEALTH_OK 2024-08-16T19:10:00.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-16T19:10:00.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:00 smithi161 ceph-mon[37576]: osdmap e92: 12 total, 12 up, 12 in 2024-08-16T19:10:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:00 smithi161 ceph-mon[37576]: overall HEALTH_OK 2024-08-16T19:10:01.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:00 smithi155 ceph-mon[37413]: osdmap e92: 12 total, 12 up, 12 in 2024-08-16T19:10:01.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:00 smithi155 ceph-mon[37413]: overall HEALTH_OK 2024-08-16T19:10:01.132 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 -- 172.21.15.62:0/2173824574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30107630 msgr2=0x7efe30109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 --2- 172.21.15.62:0/2173824574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30107630 0x7efe30109a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe28009ca0 tx=0x7efe2802a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 -- 172.21.15.62:0/2173824574 shutdown_connections 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 --2- 172.21.15.62:0/2173824574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe30107630 0x7efe30109a80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 --2- 172.21.15.62:0/2173824574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30104d00 0x7efe301070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 --2- 172.21.15.62:0/2173824574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe301023d0 0x7efe301047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 -- 172.21.15.62:0/2173824574 >> 172.21.15.62:0/2173824574 conn(0x7efe300fbfd0 msgr2=0x7efe300fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 -- 172.21.15.62:0/2173824574 shutdown_connections 2024-08-16T19:10:01.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.566+0000 7efe3487d640 1 -- 172.21.15.62:0/2173824574 wait complete. 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.567+0000 7efe3487d640 1 Processor -- start 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.567+0000 7efe3487d640 1 -- start start 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.567+0000 7efe3487d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe3487d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30104d00 0x7efe30100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe3487d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30107630 0x7efe301008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe3487d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010d5d0 con 0x7efe301023d0 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe3487d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010d450 con 0x7efe30104d00 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe3487d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010d750 con 0x7efe30107630 2024-08-16T19:10:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47208/0 (socket says 172.21.15.62:47208) 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 -- 172.21.15.62:0/689540486 learned_addr learned my addr 172.21.15.62:0/689540486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2dd9b640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30104d00 0x7efe30100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2ed9d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30107630 0x7efe301008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 -- 172.21.15.62:0/689540486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30107630 msgr2=0x7efe301008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30107630 0x7efe301008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 -- 172.21.15.62:0/689540486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30104d00 msgr2=0x7efe30100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30104d00 0x7efe30100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 -- 172.21.15.62:0/689540486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30101180 con 0x7efe301023d0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe2e59c640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe2000d950 tx=0x7efe2000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20017070 con 0x7efe301023d0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.568+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2000de80 con 0x7efe301023d0 2024-08-16T19:10:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.569+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2000f620 con 0x7efe301023d0 2024-08-16T19:10:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.569+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301a1d00 con 0x7efe301023d0 2024-08-16T19:10:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.569+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301a21c0 con 0x7efe301023d0 2024-08-16T19:10:01.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.570+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300fdac0 con 0x7efe301023d0 2024-08-16T19:10:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.571+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe20009c30 con 0x7efe301023d0 2024-08-16T19:10:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.572+0000 7efe1f7fe640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 0x7efe180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.572+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7efe200cefb0 con 0x7efe301023d0 2024-08-16T19:10:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.572+0000 7efe2dd9b640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 0x7efe180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:01.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.572+0000 7efe2dd9b640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 0x7efe180a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe30101ab0 tx=0x7efe2400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:01.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.576+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efe2009bb60 con 0x7efe301023d0 2024-08-16T19:10:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.764+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7efe30062360 con 0x7efe180a2b80 2024-08-16T19:10:01.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.770+0000 7efe1f7fe640 1 -- 172.21.15.62:0/689540486 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7efe30062360 con 0x7efe180a2b80 2024-08-16T19:10:01.771 INFO:teuthology.orchestra.run.smithi062.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-16T19:10:01.771 INFO:teuthology.orchestra.run.smithi062.stdout:mds.all 12/12 60s ago 2m smithi062=a;smithi062=d;smithi062=g;smithi062=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-08-16T19:10:01.771 INFO:teuthology.orchestra.run.smithi062.stdout:mgr 3/3 60s ago 7m smithi062=x;smithi155=y;smithi161=z;count:3 2024-08-16T19:10:01.771 INFO:teuthology.orchestra.run.smithi062.stdout:mon 3/3 60s ago 7m smithi062:172.21.15.62=a;smithi155:172.21.15.155=b;smithi161:172.21.15.161=c;count:3 2024-08-16T19:10:01.771 INFO:teuthology.orchestra.run.smithi062.stdout:osd 12 60s ago - 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 msgr2=0x7efe180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 0x7efe180a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe30101ab0 tx=0x7efe2400a2d0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 msgr2=0x7efe30101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe2000d950 tx=0x7efe2000dd50 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 shutdown_connections 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe30107630 0x7efe301008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7efe180a2b80 0x7efe180a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe30104d00 0x7efe30100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 --2- 172.21.15.62:0/689540486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe301023d0 0x7efe30101d30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 >> 172.21.15.62:0/689540486 conn(0x7efe300fbfd0 msgr2=0x7efe30102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 shutdown_connections 2024-08-16T19:10:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:01.772+0000 7efe3487d640 1 -- 172.21.15.62:0/689540486 wait complete. 2024-08-16T19:10:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:01 smithi161 ceph-mon[37576]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-08-16T19:10:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:01 smithi161 ceph-mon[37576]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:01 smithi062 ceph-mon[32993]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-08-16T19:10:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:01 smithi062 ceph-mon[32993]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:01 smithi155 ceph-mon[37413]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-08-16T19:10:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:01 smithi155 ceph-mon[37413]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:02.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-16T19:10:02.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:03.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.242+0000 7f0080936640 1 -- 172.21.15.62:0/1920685025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c101930 msgr2=0x7f007c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:03.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.242+0000 7f0080936640 1 --2- 172.21.15.62:0/1920685025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c101930 0x7f007c101d30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f006c009900 tx=0x7f006c02a410 comp rx=0 tx=0).stop 2024-08-16T19:10:03.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 -- 172.21.15.62:0/1920685025 shutdown_connections 2024-08-16T19:10:03.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 --2- 172.21.15.62:0/1920685025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c1034f0 0x7f007c109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 --2- 172.21.15.62:0/1920685025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c102b30 0x7f007c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 --2- 172.21.15.62:0/1920685025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c101930 0x7f007c101d30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 -- 172.21.15.62:0/1920685025 >> 172.21.15.62:0/1920685025 conn(0x7f007c0fd0c0 msgr2=0x7f007c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 -- 172.21.15.62:0/1920685025 shutdown_connections 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 -- 172.21.15.62:0/1920685025 wait complete. 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 Processor -- start 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 -- start start 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.243+0000 7f0080936640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c101930 0x7f007c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0080936640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c102b30 0x7f007c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0080936640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0080936640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d610 con 0x7f007c1034f0 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0080936640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d490 con 0x7f007c101930 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0080936640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f007c10d790 con 0x7f007c102b30 2024-08-16T19:10:03.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47234/0 (socket says 172.21.15.62:47234) 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 -- 172.21.15.62:0/3423619865 learned_addr learned my addr 172.21.15.62:0/3423619865 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007a59c640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c101930 0x7f007c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f0079d9b640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c102b30 0x7f007c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 -- 172.21.15.62:0/3423619865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c102b30 msgr2=0x7f007c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c102b30 0x7f007c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 -- 172.21.15.62:0/3423619865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c101930 msgr2=0x7f007c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c101930 0x7f007c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 -- 172.21.15.62:0/3423619865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c19f870 con 0x7f007c1034f0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f007ad9d640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f007400eaa0 tx=0x7f007400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.244+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007400cdb0 con 0x7f007c1034f0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.245+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c19fb60 con 0x7f007c1034f0 2024-08-16T19:10:03.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.245+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0074005020 con 0x7f007c1034f0 2024-08-16T19:10:03.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.245+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00740116b0 con 0x7f007c1034f0 2024-08-16T19:10:03.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.245+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1a0070 con 0x7f007c1034f0 2024-08-16T19:10:03.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.246+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c108d30 con 0x7f007c1034f0 2024-08-16T19:10:03.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.247+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0074005560 con 0x7f007c1034f0 2024-08-16T19:10:03.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.247+0000 7f006b7fe640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 0x7f00640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:03.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.248+0000 7f007a59c640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 0x7f00640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:03.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.248+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f00740cf660 con 0x7f007c1034f0 2024-08-16T19:10:03.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.248+0000 7f007a59c640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 0x7f00640a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f006c02aa00 tx=0x7f006c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:03.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.252+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0074016030 con 0x7f007c1034f0 2024-08-16T19:10:03.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.432+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f007c062360 con 0x7f00640a2b80 2024-08-16T19:10:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.434+0000 7f006b7fe640 1 -- 172.21.15.62:0/3423619865 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f007c062360 con 0x7f00640a2b80 2024-08-16T19:10:03.435 INFO:teuthology.orchestra.run.smithi062.stdout:HOST ADDR LABELS STATUS 2024-08-16T19:10:03.435 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 172.21.15.62 2024-08-16T19:10:03.435 INFO:teuthology.orchestra.run.smithi062.stdout:smithi155 172.21.15.155 2024-08-16T19:10:03.435 INFO:teuthology.orchestra.run.smithi062.stdout:smithi161 172.21.15.161 2024-08-16T19:10:03.435 INFO:teuthology.orchestra.run.smithi062.stdout:3 hosts in cluster 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 msgr2=0x7f00640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 0x7f00640a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f006c02aa00 tx=0x7f006c0315b0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 msgr2=0x7f007c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f007400eaa0 tx=0x7f007400ef60 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 shutdown_connections 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f007c1034f0 0x7f007c19f1d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007c102b30 0x7f007c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f00640a2b80 0x7f00640a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 --2- 172.21.15.62:0/3423619865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f007c101930 0x7f007c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:03.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 >> 172.21.15.62:0/3423619865 conn(0x7f007c0fd0c0 msgr2=0x7f007c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:03.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 shutdown_connections 2024-08-16T19:10:03.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:03.436+0000 7f0080936640 1 -- 172.21.15.62:0/3423619865 wait complete. 2024-08-16T19:10:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:03 smithi062 ceph-mon[32993]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:03 smithi062 ceph-mon[32993]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:03 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:03.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:03 smithi155 ceph-mon[37413]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:03.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:03 smithi155 ceph-mon[37413]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:03.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:03 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:03.882 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:03 smithi161 ceph-mon[37576]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:03.882 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:03 smithi161 ceph-mon[37576]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:03.882 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:03 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:04.059 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-16T19:10:04.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:04 smithi155 ceph-mon[37413]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:04.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:04 smithi161 ceph-mon[37576]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:04.973 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:04.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:04 smithi062 ceph-mon[32993]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:05.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 -- 172.21.15.62:0/1384137423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620100840 msgr2=0x7fc620100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 --2- 172.21.15.62:0/1384137423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620100840 0x7fc620100c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002a410 comp rx=0 tx=0).stop 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 -- 172.21.15.62:0/1384137423 shutdown_connections 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 --2- 172.21.15.62:0/1384137423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620102400 0x7fc620108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 --2- 172.21.15.62:0/1384137423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620101a40 0x7fc620101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 --2- 172.21.15.62:0/1384137423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620100840 0x7fc620100c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 -- 172.21.15.62:0/1384137423 >> 172.21.15.62:0/1384137423 conn(0x7fc6200fbfd0 msgr2=0x7fc6200fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 -- 172.21.15.62:0/1384137423 shutdown_connections 2024-08-16T19:10:05.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 -- 172.21.15.62:0/1384137423 wait complete. 2024-08-16T19:10:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.435+0000 7fc625d11640 1 Processor -- start 2024-08-16T19:10:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 -- start start 2024-08-16T19:10:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 0x7fc62006b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:05.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620102400 0x7fc62019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010c4a0 con 0x7fc620101a40 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010c320 con 0x7fc620100840 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc625d11640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010c620 con 0x7fc620102400 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47260/0 (socket says 172.21.15.62:47260) 2024-08-16T19:10:05.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 -- 172.21.15.62:0/4202426545 learned_addr learned my addr 172.21.15.62:0/4202426545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61f7fe640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 0x7fc62006b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61ffff640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620102400 0x7fc62019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 -- 172.21.15.62:0/4202426545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620102400 msgr2=0x7fc62019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620102400 0x7fc62019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 -- 172.21.15.62:0/4202426545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 msgr2=0x7fc62006b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 0x7fc62006b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61effd640 1 -- 172.21.15.62:0/4202426545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019f8c0 con 0x7fc620101a40 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.436+0000 7fc61f7fe640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 0x7fc62006b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc61effd640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc61400d950 tx=0x7fc61400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614017030 con 0x7fc620101a40 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019fb50 con 0x7fc620101a40 2024-08-16T19:10:05.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc614004020 con 0x7fc620101a40 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a00e0 con 0x7fc620101a40 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.437+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140047a0 con 0x7fc620101a40 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.438+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620107c90 con 0x7fc620101a40 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.439+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6140041d0 con 0x7fc620101a40 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.440+0000 7fc61cff9640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 0x7fc6080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:05.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.440+0000 7fc61f7fe640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 0x7fc6080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:05.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.440+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fc6140cf3a0 con 0x7fc620101a40 2024-08-16T19:10:05.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.440+0000 7fc61f7fe640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 0x7fc6080a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc610005ab0 tx=0x7fc610000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:05.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.443+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc61409bff0 con 0x7fc620101a40 2024-08-16T19:10:05.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.652+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc620062360 con 0x7fc6080a2b80 2024-08-16T19:10:05.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:05 smithi062 ceph-mon[32993]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:05.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.653 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:05 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.654+0000 7fc61cff9640 1 -- 172.21.15.62:0/4202426545 <== mgr.14150 v2:172.21.15.62:6800/727495888 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc620062360 con 0x7fc6080a2b80 2024-08-16T19:10:05.655 INFO:teuthology.orchestra.run.smithi062.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-16T19:10:05.655 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-16T19:10:05.655 INFO:teuthology.orchestra.run.smithi062.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-16T19:10:05.655 INFO:teuthology.orchestra.run.smithi062.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 msgr2=0x7fc6080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 0x7fc6080a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc610005ab0 tx=0x7fc610000bc0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 msgr2=0x7fc62019c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc61400d950 tx=0x7fc61400de10 comp rx=0 tx=0).stop 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 shutdown_connections 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc620102400 0x7fc62019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc620101a40 0x7fc62019c990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc6080a2b80 0x7fc6080a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 --2- 172.21.15.62:0/4202426545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc620100840 0x7fc62006b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:05.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.656+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 >> 172.21.15.62:0/4202426545 conn(0x7fc6200fbfd0 msgr2=0x7fc620104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:05.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.657+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 shutdown_connections 2024-08-16T19:10:05.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:05.657+0000 7fc625d11640 1 -- 172.21.15.62:0/4202426545 wait complete. 2024-08-16T19:10:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:05 smithi161 ceph-mon[37576]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:05 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:05 smithi155 ceph-mon[37413]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:05 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:06.369 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-16T19:10:06.378 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:10:06.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-16T19:10:06.643 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:07.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:07.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:07.019 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:07.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:07.020 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:06 smithi062 ceph-mon[32993]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 -- 172.21.15.62:0/2392305219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc1034f0 msgr2=0x7f47bc109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 --2- 172.21.15.62:0/2392305219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc1034f0 0x7f47bc109d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009ca0 tx=0x7f47b402a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 -- 172.21.15.62:0/2392305219 shutdown_connections 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 --2- 172.21.15.62:0/2392305219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc1034f0 0x7f47bc109d80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 --2- 172.21.15.62:0/2392305219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 --2- 172.21.15.62:0/2392305219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc101930 0x7f47bc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 -- 172.21.15.62:0/2392305219 >> 172.21.15.62:0/2392305219 conn(0x7f47bc0fd0c0 msgr2=0x7f47bc0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 -- 172.21.15.62:0/2392305219 shutdown_connections 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.101+0000 7f47c36b8640 1 -- 172.21.15.62:0/2392305219 wait complete. 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 Processor -- start 2024-08-16T19:10:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 -- start start 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 0x7f47bc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc10d590 con 0x7f47bc101930 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc10d410 con 0x7f47bc102b30 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c36b8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc10d710 con 0x7f47bc1034f0 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c1454640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c1454640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47272/0 (socket says 172.21.15.62:47272) 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.102+0000 7f47c1454640 1 -- 172.21.15.62:0/1030139288 learned_addr learned my addr 172.21.15.62:0/1030139288 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1c55640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 0x7f47bc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c0c53640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 -- 172.21.15.62:0/1030139288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 msgr2=0x7f47bc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 0x7f47bc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 -- 172.21.15.62:0/1030139288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 msgr2=0x7f47bc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 -- 172.21.15.62:0/1030139288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc19f7e0 con 0x7f47bc101930 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1c55640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 0x7f47bc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c0c53640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c1454640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d770 tx=0x7f47b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8017070 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8009c30 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc19fad0 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8003dd0 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.103+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc1a0060 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.104+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc108d30 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.106+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f47b8003f70 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.106+0000 7f47b27fc640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 0x7f47a80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.106+0000 7f47c0c53640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 0x7f47a80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.106+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f47b80cf190 con 0x7f47bc101930 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.107+0000 7f47c0c53640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 0x7f47a80a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f47bc1990d0 tx=0x7f47ac005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.109+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47b8009de0 con 0x7f47bc101930 2024-08-16T19:10:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.322+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f47bc02d7f0 con 0x7f47bc101930 2024-08-16T19:10:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.322+0000 7f47b27fc640 1 -- 172.21.15.62:0/1030139288 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3140 (secure 0 0 0) 0x7f47b809bdc0 con 0x7f47bc101930 2024-08-16T19:10:07.323 INFO:teuthology.orchestra.run.smithi062.stdout:e27 2024-08-16T19:10:07.323 INFO:teuthology.orchestra.run.smithi062.stdout:btime 2024-08-16T19:08:48:421739+0000 2024-08-16T19:10:07.323 INFO:teuthology.orchestra.run.smithi062.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-16T19:10:07.323 INFO:teuthology.orchestra.run.smithi062.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:legacy client fscid: 1 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:Filesystem 'cephfs' (1) 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:fs_name cephfs 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 27 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-08-16T19:08:05.356964+0000 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-08-16T19:08:48.421714+0000 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:tableserver 0 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:root 0 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:session_timeout 300 2024-08-16T19:10:07.324 INFO:teuthology.orchestra.run.smithi062.stdout:session_autoclose 300 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:max_file_size 1099511627776 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:max_xattr_size 65536 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:required_client_features {} 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure 0 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:last_failure_osd_epoch 0 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:max_mds 5 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:in 0,1,2,3,4 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:up {0=24476,1=24469,2=24451,3=24407,4=24400} 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:failed 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:damaged 2024-08-16T19:10:07.325 INFO:teuthology.orchestra.run.smithi062.stdout:stopped 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:data_pools [3,4] 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:metadata_pool 2 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:inline_data disabled 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:balancer 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:bal_rank_mask -1 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:standby_count_wanted 0 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:qdb_cluster leader: 24476 members: 24476,24469,24451,24407,24400 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b{0:24476} state up:active seq 4 addr [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.h{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/3410199866,v1:172.21.15.155:6837/3410199866] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.i{1:24469} state up:active seq 16 addr [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.326 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.f{1:24439} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/3346321531,v1:172.21.15.161:6835/3346321531] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c{2:24451} state up:active seq 18 addr [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.e{2:24431} state up:standby-replay seq 1 addr [v2:172.21.15.155:6834/3063588307,v1:172.21.15.155:6835/3063588307] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.k{3:24407} state up:active seq 22 addr [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a{3:14595} state up:standby-replay seq 1 addr [v2:172.21.15.62:6840/3131957026,v1:172.21.15.62:6841/3131957026] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.l{4:24400} state up:active seq 23 addr [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.g{4:14583} state up:standby-replay seq 1 addr [v2:172.21.15.62:6838/659126409,v1:172.21.15.62:6839/659126409] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:Standby daemons: 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.62:6834/1364784278,v1:172.21.15.62:6835/1364784278] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.62:6836/3300090633,v1:172.21.15.62:6837/3300090633] compat {c=[1],r=[1],i=[1fff]}] 2024-08-16T19:10:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:10:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 msgr2=0x7f47a80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 0x7f47a80a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f47bc1990d0 tx=0x7f47ac005e50 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 msgr2=0x7f47bc197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d770 tx=0x7f47b800dc30 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 shutdown_connections 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f47bc1034f0 0x7f47bc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f47a80a2c30 0x7f47a80a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47bc102b30 0x7f47bc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 --2- 172.21.15.62:0/1030139288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47bc101930 0x7f47bc197bb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 >> 172.21.15.62:0/1030139288 conn(0x7f47bc0fd0c0 msgr2=0x7f47bc105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 shutdown_connections 2024-08-16T19:10:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:07.324+0000 7f47c36b8640 1 -- 172.21.15.62:0/1030139288 wait complete. 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:07.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:06 smithi155 ceph-mon[37413]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.1 KiB/s wr, 9 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:07.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:06 smithi161 ceph-mon[37576]: from='mgr.14150 172.21.15.62:0/4171088316' entity='mgr.x' 2024-08-16T19:10:08.047 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-16T19:10:08.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:07 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1030139288' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-16T19:10:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:07 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1030139288' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-16T19:10:08.312 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:07 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1030139288' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 -- 172.21.15.62:0/744239778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 msgr2=0x7fdbd4109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 --2- 172.21.15.62:0/744239778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4109d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ca0 tx=0x7fdbcc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 -- 172.21.15.62:0/744239778 shutdown_connections 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 --2- 172.21.15.62:0/744239778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4109d80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 --2- 172.21.15.62:0/744239778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 0x7fdbd4102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 --2- 172.21.15.62:0/744239778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 0x7fdbd4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 -- 172.21.15.62:0/744239778 >> 172.21.15.62:0/744239778 conn(0x7fdbd40fd0c0 msgr2=0x7fdbd40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 -- 172.21.15.62:0/744239778 shutdown_connections 2024-08-16T19:10:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.735+0000 7fdbda91f640 1 -- 172.21.15.62:0/744239778 wait complete. 2024-08-16T19:10:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 Processor -- start 2024-08-16T19:10:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 -- start start 2024-08-16T19:10:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 0x7fdbd4075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 0x7fdbd4074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:08.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410d590 con 0x7fdbd41034f0 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410d410 con 0x7fdbd4102b30 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbda91f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410d710 con 0x7fdbd4101930 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbd8ebc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbd8ebc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47284/0 (socket says 172.21.15.62:47284) 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.736+0000 7fdbd8ebc640 1 -- 172.21.15.62:0/3937346696 learned_addr learned my addr 172.21.15.62:0/3937346696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd3fff640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 0x7fdbd4075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 0x7fdbd4074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 -- 172.21.15.62:0/3937346696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 msgr2=0x7fdbd4075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 0x7fdbd4075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 -- 172.21.15.62:0/3937346696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 msgr2=0x7fdbd4074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 0x7fdbd4074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 -- 172.21.15.62:0/3937346696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd4075110 con 0x7fdbd41034f0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd8ebc640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005e30 tx=0x7fdbcc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc003c30 con 0x7fdbd41034f0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd41a1c30 con 0x7fdbd41034f0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc003dd0 con 0x7fdbd41034f0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc02fde0 con 0x7fdbd41034f0 2024-08-16T19:10:08.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.737+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a2190 con 0x7fdbd41034f0 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.738+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4108c60 con 0x7fdbd41034f0 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.739+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbcc004050 con 0x7fdbd41034f0 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.740+0000 7fdbd1ffb640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 0x7fdbc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.740+0000 7fdbd3fff640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 0x7fdbc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.740+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fdbcc0f06e0 con 0x7fdbd41034f0 2024-08-16T19:10:08.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.741+0000 7fdbd3fff640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 0x7fdbc00a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4102990 tx=0x7fdbc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:08.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.744+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdbcc0bd3b0 con 0x7fdbd41034f0 2024-08-16T19:10:08.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.921+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fdbd4062b20 con 0x7fdbd41034f0 2024-08-16T19:10:08.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.922+0000 7fdbd1ffb640 1 -- 172.21.15.62:0/3937346696 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5672 (secure 0 0 0) 0x7fdbcc0bccb0 con 0x7fdbd41034f0 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 92 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:fsid bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-08-16T19:01:02.785646+0000 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2024-08-16T19:09:59.567346+0000 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 27 2024-08-16T19:10:08.923 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client luminous 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release squid 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-16T19:10:08.924 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.69 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 12 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6802/3542478147,v1:172.21.15.62:6803/3542478147] [v2:172.21.15.62:6804/3542478147,v1:172.21.15.62:6805/3542478147] exists,up c0cec841-7b80-4f4a-909c-74544c63b869 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/471904740,v1:172.21.15.155:6801/471904740] [v2:172.21.15.155:6802/471904740,v1:172.21.15.155:6803/471904740] exists,up 54e23d69-41c1-420c-aa46-2833c651059e 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/3525036557,v1:172.21.15.161:6801/3525036557] [v2:172.21.15.161:6802/3525036557,v1:172.21.15.161:6803/3525036557] exists,up 42c0d89e-6fb6-4a52-94d1-8198a745128f 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 24 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6810/1745722345,v1:172.21.15.62:6811/1745722345] [v2:172.21.15.62:6812/1745722345,v1:172.21.15.62:6813/1745722345] exists,up 5d4b830f-2036-4a2e-ab45-59ed8a735275 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 31 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/34440659,v1:172.21.15.155:6809/34440659] [v2:172.21.15.155:6810/34440659,v1:172.21.15.155:6811/34440659] exists,up 047fb0e0-b0ac-45b7-8cd9-23e62d1f52de 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 36 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/2837279313,v1:172.21.15.161:6809/2837279313] [v2:172.21.15.161:6810/2837279313,v1:172.21.15.161:6811/2837279313] exists,up 5fdaca8b-0017-49e5-9731-7c7c13c5b463 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 43 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6818/3718256322,v1:172.21.15.62:6819/3718256322] [v2:172.21.15.62:6820/3718256322,v1:172.21.15.62:6821/3718256322] exists,up a966574b-ec4f-4f19-848c-a755ecbda258 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/785303840,v1:172.21.15.155:6817/785303840] [v2:172.21.15.155:6818/785303840,v1:172.21.15.155:6819/785303840] exists,up 61e8cc89-9d50-46d6-8d08-9762b73cdf70 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.8 up in weight 1 up_from 55 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2842601651,v1:172.21.15.161:6817/2842601651] [v2:172.21.15.161:6818/2842601651,v1:172.21.15.161:6819/2842601651] exists,up 31f59120-2671-4158-bf29-e39406478022 2024-08-16T19:10:08.925 INFO:teuthology.orchestra.run.smithi062.stdout:osd.9 up in weight 1 up_from 63 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6826/851969557,v1:172.21.15.62:6827/851969557] [v2:172.21.15.62:6828/851969557,v1:172.21.15.62:6829/851969557] exists,up b3ae18d6-96bf-4b37-9fa6-99129e4edf74 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:osd.10 up in weight 1 up_from 68 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/1300242753,v1:172.21.15.155:6825/1300242753] [v2:172.21.15.155:6826/1300242753,v1:172.21.15.155:6827/1300242753] exists,up b50d6aad-7475-4b81-911e-a3d08459d2c7 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:osd.11 up in weight 1 up_from 76 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/1433195011,v1:172.21.15.161:6825/1433195011] [v2:172.21.15.161:6826/1433195011,v1:172.21.15.161:6827/1433195011] exists,up 25c71c88-59d5-4d2e-98dd-85072ddb3c0a 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.a [2,11] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.e [2,4] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 2.f [8,11] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.7 [3,9] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.17 [4,8] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.1d [5,9] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.21 [4,6] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.28 [5,7] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 3.2c [4,9] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.e [9,11] 2024-08-16T19:10:08.926 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.15 [5,10] 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.17 [9,8] 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:pg_upmap_items 4.18 [9,8] 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/3198450894 expires 2024-08-17T19:01:57.573528+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3183618050 expires 2024-08-17T19:01:57.573528+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/3198450894 expires 2024-08-17T19:01:57.573528+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6801/572676055 expires 2024-08-17T19:01:33.379605+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/1123206672 expires 2024-08-17T19:01:57.573528+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/4100589739 expires 2024-08-17T19:01:33.379605+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/2033319205 expires 2024-08-17T19:01:57.573528+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3894293084 expires 2024-08-17T19:01:33.379605+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:6800/572676055 expires 2024-08-17T19:01:33.379605+0000 2024-08-16T19:10:08.927 INFO:teuthology.orchestra.run.smithi062.stdout:blocklist 172.21.15.62:0/3941316438 expires 2024-08-17T19:01:33.379605+0000 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 msgr2=0x7fdbc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 0x7fdbc00a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4102990 tx=0x7fdbc4005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 msgr2=0x7fdbd4074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc005e30 tx=0x7fdbcc031ee0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 shutdown_connections 2024-08-16T19:10:08.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbd41034f0 0x7fdbd4074850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fdbc00a2b80 0x7fdbc00a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbd4102b30 0x7fdbd4074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.924+0000 7fdbda91f640 1 --2- 172.21.15.62:0/3937346696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdbd4101930 0x7fdbd4075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.925+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 >> 172.21.15.62:0/3937346696 conn(0x7fdbd40fd0c0 msgr2=0x7fdbd4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.925+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 shutdown_connections 2024-08-16T19:10:08.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:08.925+0000 7fdbda91f640 1 -- 172.21.15.62:0/3937346696 wait complete. 2024-08-16T19:10:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:08 smithi155 ceph-mon[37413]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1012 B/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:08 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3937346696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-16T19:10:09.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:08 smithi062 ceph-mon[32993]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1012 B/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:09.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:08 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3937346696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-16T19:10:09.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:08 smithi161 ceph-mon[37576]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1012 B/s wr, 12 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:09.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:08 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3937346696' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-16T19:10:09.698 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-16T19:10:09.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:10:09.982 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:10.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 -- 172.21.15.62:0/328450949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1007b0 msgr2=0x7f8f5c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:10.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/328450949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c100c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009930 tx=0x7f8f5002a420 comp rx=0 tx=0).stop 2024-08-16T19:10:10.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 -- 172.21.15.62:0/328450949 shutdown_connections 2024-08-16T19:10:10.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/328450949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c101150 0x7f8f5c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/328450949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c100c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/328450949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 -- 172.21.15.62:0/328450949 >> 172.21.15.62:0/328450949 conn(0x7f8f5c0fc4d0 msgr2=0x7f8f5c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 -- 172.21.15.62:0/328450949 shutdown_connections 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.420+0000 7f8f63cfd640 1 -- 172.21.15.62:0/328450949 wait complete. 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 Processor -- start 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 -- start start 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 0x7f8f5c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c111cd0 con 0x7f8f5c1067b0 2024-08-16T19:10:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c111b50 con 0x7f8f5c101150 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f63cfd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c111e50 con 0x7f8f5c1007b0 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f6229a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f61298640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 0x7f8f5c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f6229a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47294/0 (socket says 172.21.15.62:47294) 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.421+0000 7f8f6229a640 1 -- 172.21.15.62:0/834950470 learned_addr learned my addr 172.21.15.62:0/834950470 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f61a99640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 -- 172.21.15.62:0/834950470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c1007b0 msgr2=0x7f8f5c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 -- 172.21.15.62:0/834950470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 msgr2=0x7f8f5c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 0x7f8f5c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 -- 172.21.15.62:0/834950470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c19f330 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f61298640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 0x7f8f5c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f6229a640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400eaa0 tx=0x7f8f5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400cdb0 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f54005020 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c19f620 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540116b0 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.422+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c1a6f00 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.423+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c100c10 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.425+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f54005560 con 0x7f8f5c1067b0 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.425+0000 7f8f4effd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 0x7f8f480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.425+0000 7f8f61a99640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 0x7f8f480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:10.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.425+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f8f540cf910 con 0x7f8f5c1067b0 2024-08-16T19:10:10.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.426+0000 7f8f61a99640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 0x7f8f480a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f58005d00 tx=0x7f8f5800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:10.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.428+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8f54016030 con 0x7f8f5c1067b0 2024-08-16T19:10:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.643+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f5c19b6e0 con 0x7f8f5c1067b0 2024-08-16T19:10:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.643+0000 7f8f4effd640 1 -- 172.21.15.62:0/834950470 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f8f5c19b6e0 con 0x7f8f5c1067b0 2024-08-16T19:10:10.644 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:10.645 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:10:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 msgr2=0x7f8f480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 0x7f8f480a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f58005d00 tx=0x7f8f5800b410 comp rx=0 tx=0).stop 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 msgr2=0x7f8f5c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400eaa0 tx=0x7f8f5400ef60 comp rx=0 tx=0).stop 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 shutdown_connections 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f5c1067b0 0x7f8f5c19ec50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f8f480a2b80 0x7f8f480a5040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f5c101150 0x7f8f5c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 --2- 172.21.15.62:0/834950470 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f5c1007b0 0x7f8f5c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:10.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 >> 172.21.15.62:0/834950470 conn(0x7f8f5c0fc4d0 msgr2=0x7f8f5c10be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:10.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 shutdown_connections 2024-08-16T19:10:10.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:10.646+0000 7f8f63cfd640 1 -- 172.21.15.62:0/834950470 wait complete. 2024-08-16T19:10:11.244 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:10 smithi062 ceph-mon[32993]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 911 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:11.244 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:10 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/834950470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:11.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:10:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:10 smithi155 ceph-mon[37413]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 911 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:10 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/834950470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:10 smithi161 ceph-mon[37576]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 911 B/s wr, 11 op/s; 2 B/s, 0 objects/s recovering 2024-08-16T19:10:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:10 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/834950470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.020+0000 7f3118cde640 1 -- 172.21.15.62:0/2849327295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 msgr2=0x7f3114106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.020+0000 7f3118cde640 1 --2- 172.21.15.62:0/2849327295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114106530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3108009900 tx=0x7f310802a410 comp rx=0 tx=0).stop 2024-08-16T19:10:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.020+0000 7f3118cde640 1 -- 172.21.15.62:0/2849327295 shutdown_connections 2024-08-16T19:10:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.020+0000 7f3118cde640 1 --2- 172.21.15.62:0/2849327295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f311410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 --2- 172.21.15.62:0/2849327295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114106530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 --2- 172.21.15.62:0/2849327295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 0x7f3114100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 -- 172.21.15.62:0/2849327295 >> 172.21.15.62:0/2849327295 conn(0x7f3114076f40 msgr2=0x7f31140fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 -- 172.21.15.62:0/2849327295 shutdown_connections 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 -- 172.21.15.62:0/2849327295 wait complete. 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 Processor -- start 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 -- start start 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 0x7f3114100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.021+0000 7f3118cde640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f3114101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111d00 con 0x7f31141009f0 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111b80 con 0x7f3114106c60 2024-08-16T19:10:12.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111e80 con 0x7f3114100040 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42908/0 (socket says 172.21.15.62:42908) 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f311259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 0x7f3114100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 -- 172.21.15.62:0/1096852017 learned_addr learned my addr 172.21.15.62:0/1096852017 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3112d9d640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f3114101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 -- 172.21.15.62:0/1096852017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 msgr2=0x7f3114100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 0x7f3114100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 -- 172.21.15.62:0/1096852017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 msgr2=0x7f3114101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f3114101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 -- 172.21.15.62:0/1096852017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31141a6740 con 0x7f31141009f0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3112d9d640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f3114101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3111d9b640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3108006010 tx=0x7f31080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310803f030 con 0x7f31141009f0 2024-08-16T19:10:12.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3108003ce0 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141a69d0 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108035070 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.022+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141a6f00 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.024+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3114100420 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.025+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3108042030 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.025+0000 7f31037fe640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 0x7f30fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.026+0000 7f311259c640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 0x7f30fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.026+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f31080ef980 con 0x7f31141009f0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.026+0000 7f311259c640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 0x7f30fc0a50c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f310400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.030+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f31080bc5f0 con 0x7f31141009f0 2024-08-16T19:10:12.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.242+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3114109ea0 con 0x7f31141009f0 2024-08-16T19:10:12.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.243+0000 7f31037fe640 1 -- 172.21.15.62:0/1096852017 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11989 (secure 0 0 0) 0x7f31080bbef0 con 0x7f31141009f0 2024-08-16T19:10:12.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:12.243 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"btime":"2024-08-16T19:08:48:421739+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:08:48.421714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:10:12.244 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-08-16T19:10:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 msgr2=0x7f30fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 0x7f30fc0a50c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3104005fd0 tx=0x7f310400b390 comp rx=0 tx=0).stop 2024-08-16T19:10:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 msgr2=0x7f3114101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3108006010 tx=0x7f31080042f0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 shutdown_connections 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f30fc0a2c00 0x7f30fc0a50c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3114106c60 0x7f3114101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31141009f0 0x7f3114101500 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 --2- 172.21.15.62:0/1096852017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3114100040 0x7f3114100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 >> 172.21.15.62:0/1096852017 conn(0x7f3114076f40 msgr2=0x7f3114104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 shutdown_connections 2024-08-16T19:10:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:12.245+0000 7f3118cde640 1 -- 172.21.15.62:0/1096852017 wait complete. 2024-08-16T19:10:13.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:10:13.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:12 smithi062 ceph-mon[32993]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-08-16T19:10:13.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:12 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1096852017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:13.325 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:13.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:12 smithi155 ceph-mon[37413]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-08-16T19:10:13.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:12 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1096852017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:13.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:12 smithi161 ceph-mon[37576]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-08-16T19:10:13.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:12 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1096852017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:13.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 -- 172.21.15.62:0/2831419155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 msgr2=0x7f1430100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 --2- 172.21.15.62:0/2831419155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f1430100c10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1418009930 tx=0x7f141802a420 comp rx=0 tx=0).stop 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 -- 172.21.15.62:0/2831419155 shutdown_connections 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 --2- 172.21.15.62:0/2831419155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f143010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 --2- 172.21.15.62:0/2831419155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f1430100c10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 --2- 172.21.15.62:0/2831419155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 0x7f1430106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 -- 172.21.15.62:0/2831419155 >> 172.21.15.62:0/2831419155 conn(0x7f14300fc4d0 msgr2=0x7f14300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 -- 172.21.15.62:0/2831419155 shutdown_connections 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.747+0000 7f1435987640 1 -- 172.21.15.62:0/2831419155 wait complete. 2024-08-16T19:10:13.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 Processor -- start 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 -- start start 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f14300fd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 0x7f14300fd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1430110bb0 con 0x7f14301007b0 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1430110a30 con 0x7f1430101150 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f1435987640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1430110d30 con 0x7f14301067b0 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f142effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f142effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42926/0 (socket says 172.21.15.62:42926) 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f142effd640 1 -- 172.21.15.62:0/3422501547 learned_addr learned my addr 172.21.15.62:0/3422501547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.748+0000 7f142e7fc640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f14300fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142f7fe640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 0x7f14300fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 -- 172.21.15.62:0/3422501547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 msgr2=0x7f14300fd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 0x7f14300fd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 -- 172.21.15.62:0/3422501547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 msgr2=0x7f14300fd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f14300fd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 -- 172.21.15.62:0/3422501547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14301a6610 con 0x7f14301007b0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142e7fc640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f14300fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f142effd640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f142000d770 tx=0x7f142000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420017070 con 0x7f14301007b0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14301a6900 con 0x7f14301007b0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1420005500 con 0x7f14301007b0 2024-08-16T19:10:13.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420004e00 con 0x7f14301007b0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.749+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a6e10 con 0x7f14301007b0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.750+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430100c10 con 0x7f14301007b0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.751+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1420004fa0 con 0x7f14301007b0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.752+0000 7f1434985640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 0x7f141c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.752+0000 7f142e7fc640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 0x7f141c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.752+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f14200cee60 con 0x7f14301007b0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.753+0000 7f142e7fc640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 0x7f141c0a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1418000c00 tx=0x7f1418005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.756+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f142009ba10 con 0x7f14301007b0 2024-08-16T19:10:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.933+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1430109d50 con 0x7f14301007b0 2024-08-16T19:10:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.935+0000 7f1434985640 1 -- 172.21.15.62:0/3422501547 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24828 (secure 0 0 0) 0x7f142009b310 con 0x7f14301007b0 2024-08-16T19:10:13.936 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:13.936 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":92,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:09:59.567346+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 msgr2=0x7f141c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 0x7f141c0a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1418000c00 tx=0x7f1418005a20 comp rx=0 tx=0).stop 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 msgr2=0x7f14300fcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f142000d770 tx=0x7f142000dc30 comp rx=0 tx=0).stop 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 shutdown_connections 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14301067b0 0x7f14300fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f141c0a2b80 0x7f141c0a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1430101150 0x7f14300fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 --2- 172.21.15.62:0/3422501547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14301007b0 0x7f14300fcad0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 >> 172.21.15.62:0/3422501547 conn(0x7f14300fc4d0 msgr2=0x7f143006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 shutdown_connections 2024-08-16T19:10:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:13.937+0000 7f1435987640 1 -- 172.21.15.62:0/3422501547 wait complete. 2024-08-16T19:10:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:13 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3422501547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:13 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3422501547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:14.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:13 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3422501547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:14.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-16T19:08:05.356964+0000', 'modified': '2024-08-16T19:08:48.421714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24451, 'mds_3': 24407, 'mds_4': 24400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/659126409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 659126409}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 659126409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/3131957026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 3131957026}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 3131957026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.161:6833/1822368312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 1822368312}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 1822368312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.155:6833/2391473540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 2391473540}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 2391473540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/3063588307', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 3063588307}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 3063588307}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3346321531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3346321531}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3346321531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3410199866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3410199866}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3410199866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.161:6837/3769663917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3769663917}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3769663917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.161:6839/2650096635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2650096635}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2650096635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6839/2045382457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2045382457}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2045382457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24469, 24451, 24407, 24400]} 2024-08-16T19:10:14.721 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-16T19:10:14.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.l status 2024-08-16T19:10:14.990 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:14 smithi062 ceph-mon[32993]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-08-16T19:10:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:14 smithi155 ceph-mon[37413]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-08-16T19:10:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:14 smithi161 ceph-mon[37576]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-08-16T19:10:15.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 -- 172.21.15.62:0/297121945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a0103450 msgr2=0x7f60a010b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/297121945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a0103450 0x7f60a010b940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6090009900 tx=0x7f609002a480 comp rx=0 tx=0).stop 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 -- 172.21.15.62:0/297121945 shutdown_connections 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/297121945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a010bed0 0x7f60a010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/297121945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a0103450 0x7f60a010b940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/297121945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 -- 172.21.15.62:0/297121945 >> 172.21.15.62:0/297121945 conn(0x7f60a00fc4d0 msgr2=0x7f60a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 -- 172.21.15.62:0/297121945 shutdown_connections 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.424+0000 7f60a7ae7640 1 -- 172.21.15.62:0/297121945 wait complete. 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 Processor -- start 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 -- start start 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 0x7f60a0100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0111ca0 con 0x7f60a010bed0 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0111b20 con 0x7f60a0102b30 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a7ae7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0111e20 con 0x7f60a0103450 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a6084640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a6084640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42934/0 (socket says 172.21.15.62:42934) 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a6084640 1 -- 172.21.15.62:0/3540900136 learned_addr learned my addr 172.21.15.62:0/3540900136 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.425+0000 7f60a5082640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 0x7f60a0100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a5883640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 -- 172.21.15.62:0/3540900136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 msgr2=0x7f60a0100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 0x7f60a0100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 -- 172.21.15.62:0/3540900136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 msgr2=0x7f60a0100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 -- 172.21.15.62:0/3540900136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a01a65e0 con 0x7f60a010bed0 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a5082640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 0x7f60a0100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a5883640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a6084640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f609800cce0 tx=0x7f609800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f6096ffd640 1 -- 172.21.15.62:0/3540900136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609800f450 con 0x7f60a010bed0 2024-08-16T19:10:15.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f6096ffd640 1 -- 172.21.15.62:0/3540900136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609800fad0 con 0x7f60a010bed0 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a01a68d0 con 0x7f60a010bed0 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f6096ffd640 1 -- 172.21.15.62:0/3540900136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098018c10 con 0x7f60a010bed0 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.426+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a01a6dc0 con 0x7f60a010bed0 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.428+0000 7f60a9ee5740 1 Processor -- start 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f6096ffd640 1 -- 172.21.15.62:0/3540900136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f609800f5f0 con 0x7f60a010bed0 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 -- start start 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251bf50 0x5620a25262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a2526810 0x5620a2528c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620a2480710 con 0x5620a251b630 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620a2418770 con 0x5620a2526810 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620a247c6d0 con 0x5620a251bf50 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42940/0 (socket says 172.21.15.62:42940) 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.429+0000 7f60a5082640 1 -- 172.21.15.62:0/1033775339 learned_addr learned my addr 172.21.15.62:0/1033775339 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5883640 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251bf50 0x5620a25262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a2526810 0x5620a2528c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f6096ffd640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 0x7f608c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 -- 172.21.15.62:0/1033775339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251bf50 msgr2=0x5620a25262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251bf50 0x5620a25262d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 -- 172.21.15.62:0/1033775339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a2526810 msgr2=0x5620a2528c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a2526810 0x5620a2528c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 -- 172.21.15.62:0/1033775339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a241f3e0 con 0x5620a251b630 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a6084640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 0x7f608c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f60a5082640 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5620a2518f40 tx=0x7f6090031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4c307b5ed837e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f6096ffd640 1 -- 172.21.15.62:0/3540900136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f60980cfae0 con 0x7f60a010bed0 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f608bfff640 1 -- 172.21.15.62:0/1033775339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090035050 con 0x5620a251b630 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f608bfff640 1 -- 172.21.15.62:0/1033775339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6090031950 con 0x5620a251b630 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.430+0000 7f608bfff640 1 -- 172.21.15.62:0/1033775339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900048e0 con 0x5620a251b630 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a6084640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 0x7f608c0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6098061ec0 tx=0x7f609806be40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 -- 172.21.15.62:0/1033775339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 msgr2=0x5620a251ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5620a2518f40 tx=0x7f6090031480 comp rx=0 tx=0).stop 2024-08-16T19:10:15.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 -- 172.21.15.62:0/1033775339 shutdown_connections 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a2526810 0x5620a2528c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251bf50 0x5620a25262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/1033775339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a251ba10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.431+0000 7f60a9ee5740 1 -- 172.21.15.62:0/1033775339 >> 172.21.15.62:0/1033775339 conn(0x5620a2517bc0 msgr2=0x5620a2517fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- 172.21.15.62:0/1033775339 shutdown_connections 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- 172.21.15.62:0/1033775339 wait complete. 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- start start 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 Processor -- start 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2523c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a2526810 0x5620a2524190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620a25293e0 con 0x5620a251b630 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620a243b970 con 0x5620a251bf50 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.432+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620a2529560 con 0x5620a2526810 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a9ee5740 10 client.0 _ll_register_callbacks cb 0x5620a218f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42956/0 (socket says 172.21.15.62:42956) 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 -- 172.21.15.62:0/2844015871 learned_addr learned my addr 172.21.15.62:0/2844015871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5883640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2523c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a2526810 0x5620a2524190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 -- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a2526810 msgr2=0x5620a2524190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a2526810 0x5620a2524190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 -- 172.21.15.62:0/2844015871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 msgr2=0x5620a2523c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2523c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 -- 172.21.15.62:0/2844015871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a21f22b0 con 0x5620a251b630 2024-08-16T19:10:15.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a5082640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6090004650 tx=0x7f6090005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fce41143d28f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090051060 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f6090003800 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.433+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6090031950 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6090035040 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 10 client.15198 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620a2535040 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6090031950 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f609003abb0 con 0x5620a251b630 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:15.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 10 client.15198 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 20 client.15198 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a2537170 0x5620a25385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 4 client.15198 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620a248dc80 con 0x5620a2537170 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.434+0000 7f60a5883640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a2537170 0x5620a25385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.435+0000 7f60a5883640 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a2537170 0x5620a25385e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d471d5caa951d670 server_cookie=cc00f0d2f384f7bd in_seq=0 out_seq=0 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.435+0000 7f60897fa640 10 client.15198 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60897fa640 1 -- 172.21.15.62:0/2844015871 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620a248dc80 con 0x5620a2537170 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60897fa640 10 client.15198 handle_command_reply: tid=0 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 client.15198 shutdown 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 msgr2=0x5620a2523710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6090004650 tx=0x7f6090005530 comp rx=0 tx=0).stop 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 shutdown_connections 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a2537170 0x5620a25385e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a2526810 0x5620a2524190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2523c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/2844015871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a251b630 0x5620a2523710 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 >> 172.21.15.62:0/2844015871 conn(0x5620a2517bc0 msgr2=0x5620a251d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60897fa640 0 client.15198 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 shutdown_connections 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 1 -- 172.21.15.62:0/2844015871 wait complete. 2024-08-16T19:10:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.436+0000 7f60a9ee5740 20 client.15198 trim_cache size 0 max 16384 2024-08-16T19:10:15.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 Processor -- start 2024-08-16T19:10:15.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 -- start start 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a25209b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620a2536920 con 0x5620a2526810 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620a2536ff0 con 0x5620a251bf50 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620a25376c0 con 0x5620a251b630 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42960/0 (socket says 172.21.15.62:42960) 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a5082640 1 -- 172.21.15.62:0/4249204067 learned_addr learned my addr 172.21.15.62:0/4249204067 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.454+0000 7f60a5082640 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5883640 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a25209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 -- 172.21.15.62:0/4249204067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 msgr2=0x5620a25385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 -- 172.21.15.62:0/4249204067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 msgr2=0x5620a25209b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a25209b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 -- 172.21.15.62:0/4249204067 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a251b460 con 0x5620a2526810 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f60a5082640 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6090001eb0 tx=0x7f6090000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e282602030ec5a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f608b7fe640 1 -- 172.21.15.62:0/4249204067 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090051070 con 0x5620a2526810 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f608b7fe640 1 -- 172.21.15.62:0/4249204067 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6090031950 con 0x5620a2526810 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.455+0000 7f608b7fe640 1 -- 172.21.15.62:0/4249204067 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090003a40 con 0x5620a2526810 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- 172.21.15.62:0/4249204067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 msgr2=0x5620a2520ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6090001eb0 tx=0x7f6090000bc0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- 172.21.15.62:0/4249204067 shutdown_connections 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2520ef0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a25209b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/4249204067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- 172.21.15.62:0/4249204067 >> 172.21.15.62:0/4249204067 conn(0x5620a2517bc0 msgr2=0x5620a2522000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- 172.21.15.62:0/4249204067 shutdown_connections 2024-08-16T19:10:15.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- 172.21.15.62:0/4249204067 wait complete. 2024-08-16T19:10:15.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 -- start start 2024-08-16T19:10:15.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.456+0000 7f60a9ee5740 1 Processor -- start 2024-08-16T19:10:15.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2520df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620a2538460 con 0x5620a2526810 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620a2537d90 con 0x5620a251bf50 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620a2538b30 con 0x5620a251b630 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a9ee5740 10 client.0 _ll_register_callbacks cb 0x7f6090031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42968/0 (socket says 172.21.15.62:42968) 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 -- 172.21.15.62:0/471880072 learned_addr learned my addr 172.21.15.62:0/471880072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:15.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5883640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2520df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 -- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 msgr2=0x5620a25208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 -- 172.21.15.62:0/471880072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 msgr2=0x5620a2520df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2520df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 -- 172.21.15.62:0/471880072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a2513130 con 0x5620a2526810 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60a5082640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5620a2521c10 tx=0x7f6090031ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d80b15401f1f041 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:15.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.457+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090063060 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f6090003800 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6090031950 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6090051070 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 10 client.15204 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620a25a1a90 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6090031950 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f609003a810 con 0x5620a2526810 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 10 client.15204 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 20 client.15204 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a25a6010 0x5620a25a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 4 client.15204 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:15.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.458+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "status"}) -- 0x5620a2488190 con 0x5620a25a6010 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a5883640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a25a6010 0x5620a25a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a5883640 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a25a6010 0x5620a25a4d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca486450844590c0 server_cookie=d32ab887e6ca5868 in_seq=0 out_seq=0 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60897fa640 10 client.15204 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60897fa640 1 -- 172.21.15.62:0/471880072 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5620a2488190 con 0x5620a25a6010 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60897fa640 10 client.15204 handle_command_reply: tid=0 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 client.15204 shutdown 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 msgr2=0x5620a2521330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5620a2521c10 tx=0x7f6090031ed0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 shutdown_connections 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x5620a25a6010 0x5620a25a4d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620a2526810 0x5620a2521330 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620a251bf50 0x5620a2520df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60897fa640 0 client.15204 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:15.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 --2- 172.21.15.62:0/471880072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620a251b630 0x5620a25208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.459+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 >> 172.21.15.62:0/471880072 conn(0x5620a2517bc0 msgr2=0x5620a25317e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 shutdown_connections 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a9ee5740 1 -- 172.21.15.62:0/471880072 wait complete. 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a9ee5740 20 client.15204 trim_cache size 0 max 16384 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24400, 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:15.466 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 88.048275481999994, 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 92, 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 172.287911517 2024-08-16T19:10:15.467 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 msgr2=0x7f608c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 0x7f608c0a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6098061ec0 tx=0x7f609806be40 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 msgr2=0x7f60a0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f609800cce0 tx=0x7f609800a620 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 shutdown_connections 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60a010bed0 0x7f60a0101210 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60a0103450 0x7f60a0100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f608c0a2b80 0x7f608c0a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 --2- 172.21.15.62:0/3540900136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60a0102b30 0x7f60a0100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 >> 172.21.15.62:0/3540900136 conn(0x7f60a00fc4d0 msgr2=0x7f60a00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:15.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 shutdown_connections 2024-08-16T19:10:15.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:15.460+0000 7f60a7ae7640 1 -- 172.21.15.62:0/3540900136 wait complete. 2024-08-16T19:10:16.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.k status 2024-08-16T19:10:16.365 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 -- 172.21.15.62:0/989763049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c102a70 msgr2=0x7f519c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 --2- 172.21.15.62:0/989763049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c102a70 0x7f519c102e50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5198009900 tx=0x7f519802a480 comp rx=0 tx=0).stop 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 -- 172.21.15.62:0/989763049 shutdown_connections 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 --2- 172.21.15.62:0/989763049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c10bea0 0x7f519c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 --2- 172.21.15.62:0/989763049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c103390 0x7f519c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 --2- 172.21.15.62:0/989763049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c102a70 0x7f519c102e50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 -- 172.21.15.62:0/989763049 >> 172.21.15.62:0/989763049 conn(0x7f519c0fc4d0 msgr2=0x7f519c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 -- 172.21.15.62:0/989763049 shutdown_connections 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.876+0000 7f51a39a8640 1 -- 172.21.15.62:0/989763049 wait complete. 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 Processor -- start 2024-08-16T19:10:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 -- start start 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c102a70 0x7f519c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c10bea0 0x7f519c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f519c111ca0 con 0x7f519c103390 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f519c111b20 con 0x7f519c10bea0 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a39a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f519c111e20 con 0x7f519c102a70 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a0f43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a0f43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42990/0 (socket says 172.21.15.62:42990) 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a0f43640 1 -- 172.21.15.62:0/1243403379 learned_addr learned my addr 172.21.15.62:0/1243403379 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.877+0000 7f51a1744640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c102a70 0x7f519c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a1f45640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c10bea0 0x7f519c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 -- 172.21.15.62:0/1243403379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c102a70 msgr2=0x7f519c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c102a70 0x7f519c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 -- 172.21.15.62:0/1243403379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c10bea0 msgr2=0x7f519c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c10bea0 0x7f519c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 -- 172.21.15.62:0/1243403379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c19d110 con 0x7f519c103390 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a0f43640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f519000d740 tx=0x7f519000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f518e7fc640 1 -- 172.21.15.62:0/1243403379 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017070 con 0x7f519c103390 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f518e7fc640 1 -- 172.21.15.62:0/1243403379 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519000de50 con 0x7f519c103390 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c19d400 con 0x7f519c103390 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f518e7fc640 1 -- 172.21.15.62:0/1243403379 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000f620 con 0x7f519c103390 2024-08-16T19:10:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.878+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c1a4ce0 con 0x7f519c103390 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.880+0000 7f518e7fc640 1 -- 172.21.15.62:0/1243403379 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5190009c30 con 0x7f519c103390 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 Processor -- start 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 -- start start 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c47ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b82c3d2710 con 0x55b82c46d630 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b82c36a770 con 0x55b82c46df50 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a5da6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b82c3ce6d0 con 0x55b82c478810 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f518e7fc640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 0x7f51880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a1f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a1f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43006/0 (socket says 172.21.15.62:43006) 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f51a1f45640 1 -- 172.21.15.62:0/2754358681 learned_addr learned my addr 172.21.15.62:0/2754358681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.881+0000 7f518e7fc640 1 -- 172.21.15.62:0/1243403379 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f51900ceff0 con 0x7f519c103390 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1744640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a0f43640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c47ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 -- 172.21.15.62:0/2754358681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 msgr2=0x55b82c47ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c47ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 -- 172.21.15.62:0/2754358681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 msgr2=0x55b82c4782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4782d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 -- 172.21.15.62:0/2754358681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82c3713e0 con 0x55b82c46d630 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a0f43640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c47ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 0x7f51880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51a1f45640 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5194009e60 tx=0x7f5194010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c01e392301a987 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.882+0000 7f51877fe640 1 -- 172.21.15.62:0/2754358681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194021070 con 0x55b82c46d630 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51877fe640 1 -- 172.21.15.62:0/2754358681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51940113e0 con 0x55b82c46d630 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51877fe640 1 -- 172.21.15.62:0/2754358681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194016070 con 0x55b82c46d630 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a1f45640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 0x7f51880a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5194011010 tx=0x7f5194021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 -- 172.21.15.62:0/2754358681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 msgr2=0x55b82c46da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5194009e60 tx=0x7f5194010740 comp rx=0 tx=0).stop 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 -- 172.21.15.62:0/2754358681 shutdown_connections 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c47ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 --2- 172.21.15.62:0/2754358681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c46da10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 -- 172.21.15.62:0/2754358681 >> 172.21.15.62:0/2754358681 conn(0x55b82c469bc0 msgr2=0x55b82c469fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 -- 172.21.15.62:0/2754358681 shutdown_connections 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.883+0000 7f51a5da6740 1 -- 172.21.15.62:0/2754358681 wait complete. 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 -- start start 2024-08-16T19:10:16.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 Processor -- start 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c475750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 0x55b82c4761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b82c47b3e0 con 0x55b82c46df50 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b82c38d970 con 0x55b82c478810 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b82c47b560 con 0x55b82c46d630 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a5da6740 10 client.0 _ll_register_callbacks cb 0x55b82c0e17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a0f43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a0f43640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43018/0 (socket says 172.21.15.62:43018) 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.884+0000 7f51a0f43640 1 -- 172.21.15.62:0/3131367580 learned_addr learned my addr 172.21.15.62:0/3131367580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a1744640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c475750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a1f45640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 0x55b82c4761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 -- 172.21.15.62:0/3131367580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 msgr2=0x55b82c475750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c475750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 -- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 msgr2=0x55b82c4761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 0x55b82c4761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 -- 172.21.15.62:0/3131367580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82c1442b0 con 0x55b82c46df50 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a1f45640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 0x55b82c4761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a0f43640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f51900cc740 tx=0x7f519009aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed39a185105384ea server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f5184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190020040 con 0x55b82c46df50 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f519009a8f0 con 0x55b82c46df50 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51940113e0 con 0x55b82c46df50 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f51900a4110 con 0x55b82c46df50 2024-08-16T19:10:16.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a5da6740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.885+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b82c48b3c0 con 0x55b82c46df50 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51940113e0 con 0x55b82c46df50 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f519009a360 con 0x55b82c46df50 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 10 client.15216 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 20 client.15216 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c48da20 0x55b82c48ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 4 client.15216 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82c3dfc80 con 0x55b82c48da20 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.886+0000 7f51a1744640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c48da20 0x55b82c48ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f51a1744640 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c48da20 0x55b82c48ee90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a952fcce695d7a11 server_cookie=ad6895ef24f76326 in_seq=0 out_seq=0 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f5184ff9640 10 client.15216 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f5184ff9640 1 -- 172.21.15.62:0/3131367580 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b82c3dfc80 con 0x55b82c48da20 2024-08-16T19:10:16.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f5184ff9640 10 client.15216 handle_command_reply: tid=0 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f51a5da6740 1 client.15216 shutdown 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 msgr2=0x55b82c475c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.887+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f51900cc740 tx=0x7f519009aec0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 shutdown_connections 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c478810 0x55b82c4761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46df50 0x55b82c475c90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c48da20 0x55b82c48ee90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3131367580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c475750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 >> 172.21.15.62:0/3131367580 conn(0x55b82c469bc0 msgr2=0x55b82c46f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f5184ff9640 0 client.15216 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 shutdown_connections 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 1 -- 172.21.15.62:0/3131367580 wait complete. 2024-08-16T19:10:16.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.888+0000 7f51a5da6740 20 client.15216 trim_cache size 0 max 16384 2024-08-16T19:10:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 Processor -- start 2024-08-16T19:10:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 -- start start 2024-08-16T19:10:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c48e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c470090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48d1d0 con 0x55b82c478810 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48d8a0 con 0x55b82c46df50 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a5da6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48df70 con 0x55b82c46d630 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a1f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a1f45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43020/0 (socket says 172.21.15.62:43020) 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a1f45640 1 -- 172.21.15.62:0/3572015968 learned_addr learned my addr 172.21.15.62:0/3572015968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a0f43640 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c470090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.906+0000 7f51a1744640 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c48e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 -- 172.21.15.62:0/3572015968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 msgr2=0x55b82c48e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c48e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 -- 172.21.15.62:0/3572015968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 msgr2=0x55b82c470090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c470090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 -- 172.21.15.62:0/3572015968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82c473ca0 con 0x55b82c478810 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f51a1f45640 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5194009db0 tx=0x7f5194024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8527d381df6e9a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f5186ffd640 1 -- 172.21.15.62:0/3572015968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194016040 con 0x55b82c478810 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f5186ffd640 1 -- 172.21.15.62:0/3572015968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519401b990 con 0x55b82c478810 2024-08-16T19:10:16.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.907+0000 7f5186ffd640 1 -- 172.21.15.62:0/3572015968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194005620 con 0x55b82c478810 2024-08-16T19:10:16.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- 172.21.15.62:0/3572015968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 msgr2=0x55b82c474020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5194009db0 tx=0x7f5194024000 comp rx=0 tx=0).stop 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- 172.21.15.62:0/3572015968 shutdown_connections 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c478810 0x55b82c474020 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c470090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 --2- 172.21.15.62:0/3572015968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c46d630 0x55b82c48e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- 172.21.15.62:0/3572015968 >> 172.21.15.62:0/3572015968 conn(0x55b82c469bc0 msgr2=0x55b82c470a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- 172.21.15.62:0/3572015968 shutdown_connections 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- 172.21.15.62:0/3572015968 wait complete. 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 -- start start 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 Processor -- start 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.908+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c490f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48ed10 con 0x55b82c46d630 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48e640 con 0x55b82c46df50 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b82c48f3e0 con 0x55b82c478810 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 10 client.0 _ll_register_callbacks cb 0x7f5190097080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43026/0 (socket says 172.21.15.62:43026) 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 -- 172.21.15.62:0/208748826 learned_addr learned my addr 172.21.15.62:0/208748826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a0f43640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1f45640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c490f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 -- 172.21.15.62:0/208748826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 msgr2=0x55b82c490f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c490f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 -- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 msgr2=0x55b82c4909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 -- 172.21.15.62:0/208748826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82c4914b0 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a1744640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5198009850 tx=0x7f5198005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d797ce5f009956f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f5184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198035040 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f5198005020 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.909+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519401b990 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5198057050 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b82c495fa0 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f519401b990 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f51980035f0 con 0x55b82c46d630 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:16.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 10 client.15222 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 20 client.15222 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c4fcb50 0x55b82c4fb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 4 client.15222 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "status"}) -- 0x55b82c3da190 con 0x55b82c4fcb50 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.910+0000 7f51a0f43640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c4fcb50 0x55b82c4fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a0f43640 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c4fcb50 0x55b82c4fb860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=695aa046d4381c30 server_cookie=5622a520a084bbd8 in_seq=0 out_seq=0 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f5184ff9640 10 client.15222 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f5184ff9640 1 -- 172.21.15.62:0/208748826 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b82c3da190 con 0x55b82c4fcb50 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f5184ff9640 10 client.15222 handle_command_reply: tid=0 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 client.15222 shutdown 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 msgr2=0x55b82c497660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5198009850 tx=0x7f5198005e70 comp rx=0 tx=0).stop 2024-08-16T19:10:16.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 shutdown_connections 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55b82c4fcb50 0x55b82c4fb860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b82c478810 0x55b82c490f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b82c46df50 0x55b82c4909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 --2- 172.21.15.62:0/208748826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b82c46d630 0x55b82c497660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 >> 172.21.15.62:0/208748826 conn(0x55b82c469bc0 msgr2=0x55b82c473eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f5184ff9640 0 client.15222 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 shutdown_connections 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 1 -- 172.21.15.62:0/208748826 wait complete. 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.911+0000 7f51a5da6740 20 client.15222 trim_cache size 0 max 16384 2024-08-16T19:10:16.917 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24407, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 90.505922120999998, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 92, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 171.07876324700001 2024-08-16T19:10:16.918 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 msgr2=0x7f51880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 0x7f51880a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5194011010 tx=0x7f5194021000 comp rx=0 tx=0).stop 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 msgr2=0x7f519c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f519000d740 tx=0x7f519000dc00 comp rx=0 tx=0).stop 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 shutdown_connections 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f51880a2b80 0x7f51880a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f519c10bea0 0x7f519c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f519c103390 0x7f519c1986a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 --2- 172.21.15.62:0/1243403379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f519c102a70 0x7f519c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 >> 172.21.15.62:0/1243403379 conn(0x7f519c0fc4d0 msgr2=0x7f519c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 shutdown_connections 2024-08-16T19:10:16.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:16.912+0000 7f51a39a8640 1 -- 172.21.15.62:0/1243403379 wait complete. 2024-08-16T19:10:17.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:16 smithi062 ceph-mon[32993]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:17.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:16 smithi155 ceph-mon[37413]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:17.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:16 smithi161 ceph-mon[37576]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:17.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-16T19:10:17.999 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:18.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 -- 172.21.15.62:0/1087144610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0102a70 msgr2=0x7fb0b0102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 --2- 172.21.15.62:0/1087144610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0102a70 0x7fb0b0102e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009900 tx=0x7fb0a002a480 comp rx=0 tx=0).stop 2024-08-16T19:10:18.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 -- 172.21.15.62:0/1087144610 shutdown_connections 2024-08-16T19:10:18.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 --2- 172.21.15.62:0/1087144610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 0x7fb0b010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 --2- 172.21.15.62:0/1087144610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0103390 0x7fb0b010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 --2- 172.21.15.62:0/1087144610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0102a70 0x7fb0b0102e50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 -- 172.21.15.62:0/1087144610 >> 172.21.15.62:0/1087144610 conn(0x7fb0b00fc4d0 msgr2=0x7fb0b00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 -- 172.21.15.62:0/1087144610 shutdown_connections 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.419+0000 7fb0b5539640 1 -- 172.21.15.62:0/1087144610 wait complete. 2024-08-16T19:10:18.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.420+0000 7fb0b5539640 1 Processor -- start 2024-08-16T19:10:18.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 -- start start 2024-08-16T19:10:18.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 0x7fb0b01006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 0x7fb0b0101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0111ce0 con 0x7fb0b0103390 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0111b60 con 0x7fb0b010bea0 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0b5539640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0111e60 con 0x7fb0b0102a70 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43038/0 (socket says 172.21.15.62:43038) 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3866400650 learned_addr learned my addr 172.21.15.62:0/3866400650 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 0x7fb0b0101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.421+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 0x7fb0b01006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3866400650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 msgr2=0x7fb0b01006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 0x7fb0b01006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3866400650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 msgr2=0x7fb0b0101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 0x7fb0b0101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3866400650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b01a66a0 con 0x7fb0b0103390 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 0x7fb0b01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b3f0 tx=0x7fb0a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb097fff640 1 -- 172.21.15.62:0/3866400650 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4015020 con 0x7fb0b0103390 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01a6990 con 0x7fb0b0103390 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb097fff640 1 -- 172.21.15.62:0/3866400650 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4011d10 con 0x7fb0b0103390 2024-08-16T19:10:18.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a6ed0 con 0x7fb0b0103390 2024-08-16T19:10:18.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.422+0000 7fb097fff640 1 -- 172.21.15.62:0/3866400650 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400f2e0 con 0x7fb0b0103390 2024-08-16T19:10:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 Processor -- start 2024-08-16T19:10:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 -- start start 2024-08-16T19:10:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb097fff640 1 -- 172.21.15.62:0/3866400650 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0a4018020 con 0x7fb0b0103390 2024-08-16T19:10:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae952d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d1cadef710 con 0x55d1cae95810 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1cad87770 con 0x55d1cae8a630 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0b7937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1cadeb6d0 con 0x55d1cae8af50 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43050/0 (socket says 172.21.15.62:43050) 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3866347360 learned_addr learned my addr 172.21.15.62:0/3866347360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:49362/0 (socket says 172.21.15.62:49362) 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.425+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3866347360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 msgr2=0x55d1cae952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3866347360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 msgr2=0x55d1cae97c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3866347360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cad8e3e0 con 0x55d1cae8a630 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb097fff640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 0x7fb0980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009f30 tx=0x7fb0a8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0b68e467fb0602 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 0x7fb0980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb097fff640 1 -- 172.21.15.62:0/3866400650 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb0a40cec70 con 0x7fb0b0103390 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0957fa640 1 -- 172.21.15.62:0/3866347360 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8016070 con 0x55d1cae8a630 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0957fa640 1 -- 172.21.15.62:0/3866347360 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8004d20 con 0x55d1cae8a630 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.426+0000 7fb0957fa640 1 -- 172.21.15.62:0/3866347360 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8012870 con 0x55d1cae8a630 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- 172.21.15.62:0/3866347360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 msgr2=0x55d1cae8aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009f30 tx=0x7fb0a8010800 comp rx=0 tx=0).stop 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 0x7fb0980a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55d1cae87a20 tx=0x7fb0a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- 172.21.15.62:0/3866347360 shutdown_connections 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae97c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3866347360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae8aa10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- 172.21.15.62:0/3866347360 >> 172.21.15.62:0/3866347360 conn(0x55d1cae86bc0 msgr2=0x55d1cae86fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- 172.21.15.62:0/3866347360 shutdown_connections 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- 172.21.15.62:0/3866347360 wait complete. 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 -- start start 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.427+0000 7fb0b7937740 1 Processor -- start 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae92c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d1cae983e0 con 0x55d1cae95810 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1cadaa970 con 0x55d1cae8a630 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1cae98560 con 0x55d1cae8af50 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0b7937740 10 client.0 _ll_register_callbacks cb 0x55d1caafe7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43064/0 (socket says 172.21.15.62:43064) 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3395238161 learned_addr learned my addr 172.21.15.62:0/3395238161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 msgr2=0x55d1cae92c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae92c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3395238161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 msgr2=0x55d1cae926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 -- 172.21.15.62:0/3395238161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cab612b0 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004c70 tx=0x7fb0a80059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c99c04011d40a24 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.428+0000 7fb092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8029060 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8004d20 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb0a80050e0 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a8030050 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1caea3fe0 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8004d20 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a800f040 con 0x55d1cae95810 2024-08-16T19:10:18.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 20 client.15234 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caea54a0 0x55d1caea76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 4 client.15234 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.429+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cadfcc80 con 0x55d1caea54a0 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.430+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caea54a0 0x55d1caea76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.430+0000 7fb092ff5640 10 client.15234 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.430+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caea54a0 0x55d1caea76a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5d84ce63fe2f5d9 server_cookie=e918913bf36f7094 in_seq=0 out_seq=0 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb092ff5640 1 -- 172.21.15.62:0/3395238161 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1cadfcc80 con 0x55d1caea54a0 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb092ff5640 10 client.15234 handle_command_reply: tid=0 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 client.15234 shutdown 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 msgr2=0x55d1cae93140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004c70 tx=0x7fb0a80059f0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 shutdown_connections 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae95810 0x55d1cae93140 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae8af50 0x55d1cae92c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caea54a0 0x55d1caea76a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3395238161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1cae926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 >> 172.21.15.62:0/3395238161 conn(0x55d1cae86bc0 msgr2=0x55d1cae85d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb092ff5640 0 client.15234 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 shutdown_connections 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 1 -- 172.21.15.62:0/3395238161 wait complete. 2024-08-16T19:10:18.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.431+0000 7fb0b7937740 20 client.15234 trim_cache size 0 max 16384 2024-08-16T19:10:18.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 Processor -- start 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 -- start start 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1caea1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea59e0 con 0x55d1cae8af50 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea60b0 con 0x55d1cae8a630 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0b7937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea6780 con 0x55d1cae95810 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0ae7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43072/0 (socket says 172.21.15.62:43072) 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3229843829 learned_addr learned my addr 172.21.15.62:0/3229843829 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.449+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1caea1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3229843829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 msgr2=0x55d1caea5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3229843829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 msgr2=0x55d1caea1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1caea1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 -- 172.21.15.62:0/3229843829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cae869f0 con 0x55d1cae8af50 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097580 tx=0x7fb0a40d2750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d570b6f7e039371d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb094ff9640 1 -- 172.21.15.62:0/3229843829 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a409aa80 con 0x55d1cae8af50 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb094ff9640 1 -- 172.21.15.62:0/3229843829 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4010eb0 con 0x55d1cae8af50 2024-08-16T19:10:18.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.450+0000 7fb094ff9640 1 -- 172.21.15.62:0/3229843829 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4009200 con 0x55d1cae8af50 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- 172.21.15.62:0/3229843829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 msgr2=0x55d1caea6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097580 tx=0x7fb0a40d2750 comp rx=0 tx=0).stop 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- 172.21.15.62:0/3229843829 shutdown_connections 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8af50 0x55d1caea6fd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- 172.21.15.62:0/3229843829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8a630 0x55d1caea1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- 172.21.15.62:0/3229843829 >> 172.21.15.62:0/3229843829 conn(0x55d1cae86bc0 msgr2=0x55d1cae86fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- 172.21.15.62:0/3229843829 shutdown_connections 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- 172.21.15.62:0/3229843829 wait complete. 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 -- start start 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 Processor -- start 2024-08-16T19:10:18.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 0x55d1caea8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.451+0000 7fb0b7937740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0b7937740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea7520 con 0x55d1cae8a630 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0b7937740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea6e50 con 0x55d1cae8af50 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0b7937740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d1caea7bf0 con 0x55d1cae95810 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0b7937740 10 client.0 _ll_register_callbacks cb 0x7fb0a40d2620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43074/0 (socket says 172.21.15.62:43074) 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 -- 172.21.15.62:0/1549195377 learned_addr learned my addr 172.21.15.62:0/1549195377 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0af7fe640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 0x55d1caea8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 -- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 msgr2=0x55d1caea8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 -- 172.21.15.62:0/1549195377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 msgr2=0x55d1caea8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 0x55d1caea8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 -- 172.21.15.62:0/1549195377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cae82130 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 0x55d1caea8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0aeffd640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002a9a0 tx=0x7fb0a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d787c5fe4793e4a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0035040 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb0a0005240 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1cadf7190 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.452+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a0060050 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d1caf0f900 con 0x55d1cae8a630 2024-08-16T19:10:18.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d1cadf7190 con 0x55d1cae8a630 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a0003710 con 0x55d1cae8a630 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 20 client.15240 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caf14530 0x55d1caf13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 4 client.15240 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "status"}) -- 0x55d1cad740e0 con 0x55d1caf14530 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.453+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caf14530 0x55d1caf13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0ae7fc640 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caf14530 0x55d1caf13240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b79ee90d65340ff1 server_cookie=ad5e8b567ea44987 in_seq=0 out_seq=0 2024-08-16T19:10:18.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb092ff5640 10 client.15240 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb092ff5640 1 -- 172.21.15.62:0/1549195377 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d1cad740e0 con 0x55d1caf14530 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb092ff5640 10 client.15240 handle_command_reply: tid=0 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 client.15240 shutdown 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 msgr2=0x55d1caea8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0a002a9a0 tx=0x7fb0a0009260 comp rx=0 tx=0).stop 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 shutdown_connections 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d1cae95810 0x55d1caea8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55d1caf14530 0x55d1caf13240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d1cae8af50 0x55d1caea8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 --2- 172.21.15.62:0/1549195377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d1cae8a630 0x55d1caea8520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 >> 172.21.15.62:0/1549195377 conn(0x55d1cae86bc0 msgr2=0x55d1cae917b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb092ff5640 0 client.15240 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:18.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 shutdown_connections 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 1 -- 172.21.15.62:0/1549195377 wait complete. 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.454+0000 7fb0b7937740 20 client.15240 trim_cache size 0 max 16384 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24451, 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:18.461 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 92.717347771999997, 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 92, 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 158.162066248 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:18.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 msgr2=0x7fb0980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 0x7fb0980a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55d1cae87a20 tx=0x7fb0a0031400 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 msgr2=0x7fb0b0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400b3f0 tx=0x7fb0a400b8b0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 shutdown_connections 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb0980a2b80 0x7fb0980a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b010bea0 0x7fb0b0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b0103390 0x7fb0b0100c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 --2- 172.21.15.62:0/3866400650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0b0102a70 0x7fb0b01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 >> 172.21.15.62:0/3866400650 conn(0x7fb0b00fc4d0 msgr2=0x7fb0b00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 shutdown_connections 2024-08-16T19:10:18.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:18.455+0000 7fb0b5539640 1 -- 172.21.15.62:0/3866400650 wait complete. 2024-08-16T19:10:19.219 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:18 smithi062 ceph-mon[32993]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-16T19:10:19.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-16T19:10:19.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:18 smithi155 ceph-mon[37413]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-16T19:10:19.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:18 smithi161 ceph-mon[37576]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-08-16T19:10:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:20.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 -- 172.21.15.62:0/2981274846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 msgr2=0x7f7714100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 --2- 172.21.15.62:0/2981274846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714100c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7708009930 tx=0x7f770802a420 comp rx=0 tx=0).stop 2024-08-16T19:10:20.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 -- 172.21.15.62:0/2981274846 shutdown_connections 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 --2- 172.21.15.62:0/2981274846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7714101150 0x7f771410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 --2- 172.21.15.62:0/2981274846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714100c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 --2- 172.21.15.62:0/2981274846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77141067b0 0x7f7714106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.992+0000 7f77198bd640 1 -- 172.21.15.62:0/2981274846 >> 172.21.15.62:0/2981274846 conn(0x7f77140fc4d0 msgr2=0x7f77140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.993+0000 7f77198bd640 1 -- 172.21.15.62:0/2981274846 shutdown_connections 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.993+0000 7f77198bd640 1 -- 172.21.15.62:0/2981274846 wait complete. 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.993+0000 7f77198bd640 1 Processor -- start 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.993+0000 7f77198bd640 1 -- start start 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7714101150 0x7f77141028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77141067b0 0x7f7714102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7714110bb0 con 0x7f77141007b0 2024-08-16T19:10:20.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7714110a30 con 0x7f77141067b0 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77198bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7714110d30 con 0x7f7714101150 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43090/0 (socket says 172.21.15.62:43090) 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 -- 172.21.15.62:0/1162554829 learned_addr learned my addr 172.21.15.62:0/1162554829 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77137fe640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77141067b0 0x7f7714102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77127fc640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7714101150 0x7f77141028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 -- 172.21.15.62:0/1162554829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7714101150 msgr2=0x7f77141028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7714101150 0x7f77141028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 -- 172.21.15.62:0/1162554829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77141067b0 msgr2=0x7f7714102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77141067b0 0x7f7714102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 -- 172.21.15.62:0/1162554829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77141a6640 con 0x7f77141007b0 2024-08-16T19:10:20.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f770400d770 tx=0x7f770400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.994+0000 7f77188bb640 1 -- 172.21.15.62:0/1162554829 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.995+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77141a6930 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.995+0000 7f77188bb640 1 -- 172.21.15.62:0/1162554829 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704005500 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.995+0000 7f77188bb640 1 -- 172.21.15.62:0/1162554829 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704004e00 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.995+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141a6e40 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.997+0000 7f77188bb640 1 -- 172.21.15.62:0/1162554829 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7704004fa0 con 0x7f77141007b0 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.997+0000 7f771bcbb740 1 Processor -- start 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.997+0000 7f771bcbb740 1 -- start start 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 0x55f2664542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266456c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f2663ae710 con 0x55f266449f50 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f266346770 con 0x55f266449630 2024-08-16T19:10:20.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f771bcbb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f2663aa6d0 con 0x55f266454810 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 0x55f2664542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77188bb640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 0x7f76fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:49414/0 (socket says 172.21.15.62:49414) 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 -- 172.21.15.62:0/2689975252 learned_addr learned my addr 172.21.15.62:0/2689975252 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77127fc640 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266456c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 0x7f76fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77188bb640 1 -- 172.21.15.62:0/1162554829 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f77040ced10 con 0x7f77141007b0 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 -- 172.21.15.62:0/2689975252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 msgr2=0x55f266456c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266456c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 -- 172.21.15.62:0/2689975252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 msgr2=0x55f2664542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 0x55f2664542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f77137fe640 1 -- 172.21.15.62:0/2689975252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26634d3e0 con 0x55f266449630 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.998+0000 7f7712ffd640 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 0x55f2664542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.999+0000 7f77137fe640 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f770c013e40 tx=0x7f770c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c954f292ab6011 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.999+0000 7f77137fe640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 0x7f76fc0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f770c00fc90 tx=0x7f770c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.999+0000 7f76f9ffb640 1 -- 172.21.15.62:0/2689975252 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c013920 con 0x55f266449630 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.999+0000 7f76f9ffb640 1 -- 172.21.15.62:0/2689975252 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c013170 con 0x55f266449630 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:19.999+0000 7f76f9ffb640 1 -- 172.21.15.62:0/2689975252 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c003d70 con 0x55f266449630 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- 172.21.15.62:0/2689975252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 msgr2=0x55f266449a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f770c013e40 tx=0x7f770c00f790 comp rx=0 tx=0).stop 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- 172.21.15.62:0/2689975252 shutdown_connections 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266456c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449f50 0x55f2664542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 --2- 172.21.15.62:0/2689975252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266449a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- 172.21.15.62:0/2689975252 >> 172.21.15.62:0/2689975252 conn(0x55f266445bc0 msgr2=0x55f266445fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- 172.21.15.62:0/2689975252 shutdown_connections 2024-08-16T19:10:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- 172.21.15.62:0/2689975252 wait complete. 2024-08-16T19:10:20.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 -- start start 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.000+0000 7f771bcbb740 1 Processor -- start 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266451700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f266451c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f266457380 con 0x55f266454810 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f266369970 con 0x55f266449630 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f266457500 con 0x55f266449f50 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f771bcbb740 10 client.0 _ll_register_callbacks cb 0x55f2660bd7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43106/0 (socket says 172.21.15.62:43106) 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 -- 172.21.15.62:0/1553994699 learned_addr learned my addr 172.21.15.62:0/1553994699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:20.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266451700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77127fc640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f266451c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 -- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 msgr2=0x55f266451c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f266451c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 -- 172.21.15.62:0/1553994699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 msgr2=0x55f266451700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266451700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 -- 172.21.15.62:0/1553994699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2661202b0 con 0x55f266454810 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266451700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f77137fe640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f770c00ffa0 tx=0x7f770c013390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231da52f521973e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f76f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.001+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c02c020 con 0x55f266454810 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f770c003c80 con 0x55f266454810 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c013170 con 0x55f266454810 2024-08-16T19:10:20.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f770c01c040 con 0x55f266454810 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f266462fe0 con 0x55f266454810 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c013170 con 0x55f266454810 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f770c0104c0 con 0x55f266454810 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 20 client.15252 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664652e0 0x55f266466750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 4 client.15252 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.002+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2663bbc80 con 0x55f2664652e0 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.003+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664652e0 0x55f266466750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.003+0000 7f7712ffd640 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664652e0 0x55f266466750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9a934a880dee1f6 server_cookie=23672be48749452a in_seq=0 out_seq=0 2024-08-16T19:10:20.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.003+0000 7f76f2ffd640 10 client.15252 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f76f2ffd640 1 -- 172.21.15.62:0/1553994699 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2663bbc80 con 0x55f2664652e0 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f76f2ffd640 10 client.15252 handle_command_reply: tid=0 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 client.15252 shutdown 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 msgr2=0x55f266452180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f770c00ffa0 tx=0x7f770c013390 comp rx=0 tx=0).stop 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 shutdown_connections 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664652e0 0x55f266466750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266454810 0x55f266452180 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f266451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 --2- 172.21.15.62:0/1553994699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449630 0x55f266451700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f76f2ffd640 0 client.15252 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 >> 172.21.15.62:0/1553994699 conn(0x55f266445bc0 msgr2=0x55f266444d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 shutdown_connections 2024-08-16T19:10:20.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 1 -- 172.21.15.62:0/1553994699 wait complete. 2024-08-16T19:10:20.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.004+0000 7f771bcbb740 20 client.15252 trim_cache size 0 max 16384 2024-08-16T19:10:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 Processor -- start 2024-08-16T19:10:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 -- start start 2024-08-16T19:10:20.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 0x55f26644c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266453560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f266464a90 con 0x55f266449630 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f266465160 con 0x55f266449f50 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f771bcbb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f266465830 con 0x55f266454810 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43116/0 (socket says 172.21.15.62:43116) 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 -- 172.21.15.62:0/3540758518 learned_addr learned my addr 172.21.15.62:0/3540758518 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f77137fe640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266453560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 -- 172.21.15.62:0/3540758518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 msgr2=0x55f266453560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266453560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f77127fc640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 0x55f26644c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 -- 172.21.15.62:0/3540758518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 msgr2=0x55f26644c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 0x55f26644c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f7712ffd640 1 -- 172.21.15.62:0/3540758518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f266441130 con 0x55f266449630 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.023+0000 7f77127fc640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 0x55f26644c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.024+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77040976d0 tx=0x7f77040bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b306bbb8349ad4 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.024+0000 7f76f97fa640 1 -- 172.21.15.62:0/3540758518 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704020040 con 0x55f266449630 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.024+0000 7f76f97fa640 1 -- 172.21.15.62:0/3540758518 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704099310 con 0x55f266449630 2024-08-16T19:10:20.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.024+0000 7f76f97fa640 1 -- 172.21.15.62:0/3540758518 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040978e0 con 0x55f266449630 2024-08-16T19:10:20.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- 172.21.15.62:0/3540758518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 msgr2=0x55f2664652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77040976d0 tx=0x7f77040bb040 comp rx=0 tx=0).stop 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- 172.21.15.62:0/3540758518 shutdown_connections 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266454810 0x55f266453560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266449f50 0x55f26644c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3540758518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f2664652e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- 172.21.15.62:0/3540758518 >> 172.21.15.62:0/3540758518 conn(0x55f266445bc0 msgr2=0x55f266452e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- 172.21.15.62:0/3540758518 shutdown_connections 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- 172.21.15.62:0/3540758518 wait complete. 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 -- start start 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.025+0000 7f771bcbb740 1 Processor -- start 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f26646a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266454810 0x55f2664c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f2664665d0 con 0x55f266449630 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55f266465f00 con 0x55f266454810 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f266466ca0 con 0x55f266449f50 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 10 client.0 _ll_register_callbacks cb 0x7f770409b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43130/0 (socket says 172.21.15.62:43130) 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 -- 172.21.15.62:0/3967255913 learned_addr learned my addr 172.21.15.62:0/3967255913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f77127fc640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f26646a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f77137fe640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266454810 0x55f2664c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 -- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 msgr2=0x55f26646a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f26646a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 -- 172.21.15.62:0/3967255913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266454810 msgr2=0x55f2664c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:20.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266454810 0x55f2664c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 -- 172.21.15.62:0/3967255913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f266452650 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f7712ffd640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7704005760 tx=0x7f770409f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2953f7ddec9b77 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f76f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040b3020 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f77040978e0 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.026+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7704099310 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7704020070 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2664cf190 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704099310 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f770409f120 con 0x55f266449630 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:10:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 20 client.15258 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664d38e0 0x55f2664d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 4 client.15258 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "status"}) -- 0x55f2663b6190 con 0x55f2664d38e0 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.027+0000 7f77127fc640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664d38e0 0x55f2664d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f77127fc640 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664d38e0 0x55f2664d25f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f73f399d52c60fc7 server_cookie=1fc115c47bf43a86 in_seq=0 out_seq=0 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f76f2ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f76f2ffd640 1 -- 172.21.15.62:0/3967255913 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f2663b6190 con 0x55f2664d38e0 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f76f2ffd640 10 client.15258 handle_command_reply: tid=0 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 client.15258 shutdown 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 msgr2=0x55f26646a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7704005760 tx=0x7f770409f6f0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 shutdown_connections 2024-08-16T19:10:20.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x55f2664d38e0 0x55f2664d25f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f266454810 0x55f2664c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f266449f50 0x55f26646a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 --2- 172.21.15.62:0/3967255913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f266449630 0x55f26646a100 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 >> 172.21.15.62:0/3967255913 conn(0x55f266445bc0 msgr2=0x55f26645ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f76f2ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.028+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 shutdown_connections 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f771bcbb740 1 -- 172.21.15.62:0/3967255913 wait complete. 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f771bcbb740 20 client.15258 trim_cache size 0 max 16384 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 95.303297584999996, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 92, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 154.464008673 2024-08-16T19:10:20.035 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 msgr2=0x7f76fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 0x7f76fc0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f770c00fc90 tx=0x7f770c01e000 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 msgr2=0x7f7714102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f770400d770 tx=0x7f770400dc30 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 shutdown_connections 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f76fc0a2b80 0x7f76fc0a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77141067b0 0x7f7714102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7714101150 0x7f77141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 --2- 172.21.15.62:0/1162554829 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77141007b0 0x7f7714102380 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.029+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 >> 172.21.15.62:0/1162554829 conn(0x7f77140fc4d0 msgr2=0x7f7714104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:20.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.030+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 shutdown_connections 2024-08-16T19:10:20.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:20.030+0000 7f77198bd640 1 -- 172.21.15.62:0/1162554829 wait complete. 2024-08-16T19:10:20.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-16T19:10:21.082 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:21.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:20 smithi062 ceph-mon[32993]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-08-16T19:10:21.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:20 smithi155 ceph-mon[37413]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-08-16T19:10:21.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:20 smithi161 ceph-mon[37576]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 -- 172.21.15.62:0/2931125760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 msgr2=0x7fb750107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 --2- 172.21.15.62:0/2931125760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb750107c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009930 tx=0x7fb74c02a420 comp rx=0 tx=0).stop 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 -- 172.21.15.62:0/2931125760 shutdown_connections 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 --2- 172.21.15.62:0/2931125760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb750102250 0x7fb75010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 --2- 172.21.15.62:0/2931125760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7501018b0 0x7fb750101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 --2- 172.21.15.62:0/2931125760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb750107c90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 -- 172.21.15.62:0/2931125760 >> 172.21.15.62:0/2931125760 conn(0x7fb7500fd5f0 msgr2=0x7fb7500ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 -- 172.21.15.62:0/2931125760 shutdown_connections 2024-08-16T19:10:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.516+0000 7fb75762a640 1 -- 172.21.15.62:0/2931125760 wait complete. 2024-08-16T19:10:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 Processor -- start 2024-08-16T19:10:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 -- start start 2024-08-16T19:10:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7501018b0 0x7fb750198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb750102250 0x7fb750198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb750111cd0 con 0x7fb7501078b0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb750111b50 con 0x7fb750102250 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb75762a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb750111e50 con 0x7fb7501018b0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34908/0 (socket says 172.21.15.62:34908) 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 -- 172.21.15.62:0/2125416933 learned_addr learned my addr 172.21.15.62:0/2125416933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb7553c6640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7501018b0 0x7fb750198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb754bc5640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb750102250 0x7fb750198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 -- 172.21.15.62:0/2125416933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7501018b0 msgr2=0x7fb750198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7501018b0 0x7fb750198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 -- 172.21.15.62:0/2125416933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb750102250 msgr2=0x7fb750198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb750102250 0x7fb750198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.517+0000 7fb755bc7640 1 -- 172.21.15.62:0/2125416933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75019d2b0 con 0x7fb7501078b0 2024-08-16T19:10:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb755bc7640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb74800b970 tx=0x7fb74800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb7467fc640 1 -- 172.21.15.62:0/2125416933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748019070 con 0x7fb7501078b0 2024-08-16T19:10:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb7467fc640 1 -- 172.21.15.62:0/2125416933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74800cb40 con 0x7fb7501078b0 2024-08-16T19:10:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75019d5a0 con 0x7fb7501078b0 2024-08-16T19:10:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb7467fc640 1 -- 172.21.15.62:0/2125416933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748007650 con 0x7fb7501078b0 2024-08-16T19:10:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.518+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501a4e80 con 0x7fb7501078b0 2024-08-16T19:10:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.520+0000 7fb7467fc640 1 -- 172.21.15.62:0/2125416933 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb74801c070 con 0x7fb7501078b0 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 Processor -- start 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 -- start start 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683f50 0x55ec5768e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec57690c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ec575e8710 con 0x55ec57683630 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ec57580770 con 0x55ec57683f50 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb759a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ec575e46d0 con 0x55ec5768e810 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb7467fc640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 0x7fb73c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34920/0 (socket says 172.21.15.62:34920) 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb754bc5640 1 -- 172.21.15.62:0/1082496846 learned_addr learned my addr 172.21.15.62:0/1082496846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb755bc7640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 0x7fb73c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb7467fc640 1 -- 172.21.15.62:0/2125416933 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb7480ce9e0 con 0x7fb7501078b0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb7553c6640 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683f50 0x55ec5768e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.521+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec57690c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 -- 172.21.15.62:0/1082496846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 msgr2=0x55ec57690c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec57690c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 -- 172.21.15.62:0/1082496846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683f50 msgr2=0x55ec5768e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683f50 0x55ec5768e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 -- 172.21.15.62:0/1082496846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec575873e0 con 0x55ec57683630 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb7400058c0 tx=0x7fb74000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dfdede59c5bfff server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb755bc7640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 0x7fb73c0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7480cc130 tx=0x7fb748004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb73b7fe640 1 -- 172.21.15.62:0/1082496846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740019020 con 0x55ec57683630 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb73b7fe640 1 -- 172.21.15.62:0/1082496846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb740010b40 con 0x55ec57683630 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.522+0000 7fb73b7fe640 1 -- 172.21.15.62:0/1082496846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74000fb80 con 0x55ec57683630 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- 172.21.15.62:0/1082496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 msgr2=0x55ec57683a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb7400058c0 tx=0x7fb74000b6b0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- 172.21.15.62:0/1082496846 shutdown_connections 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec57690c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683f50 0x55ec5768e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 --2- 172.21.15.62:0/1082496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683630 0x55ec57683a10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- 172.21.15.62:0/1082496846 >> 172.21.15.62:0/1082496846 conn(0x55ec5767fbc0 msgr2=0x55ec5767ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- 172.21.15.62:0/1082496846 shutdown_connections 2024-08-16T19:10:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- 172.21.15.62:0/1082496846 wait complete. 2024-08-16T19:10:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 -- start start 2024-08-16T19:10:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.523+0000 7fb759a28740 1 Processor -- start 2024-08-16T19:10:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec5768b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec5768bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ec576913e0 con 0x55ec5768e810 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ec575a3970 con 0x55ec57683630 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ec57691560 con 0x55ec57683f50 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb759a28740 10 client.0 _ll_register_callbacks cb 0x55ec572f77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34928/0 (socket says 172.21.15.62:34928) 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 -- 172.21.15.62:0/1257838345 learned_addr learned my addr 172.21.15.62:0/1257838345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb7553c6640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec5768bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec5768b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 -- 172.21.15.62:0/1257838345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 msgr2=0x55ec5768bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec5768bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 -- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 msgr2=0x55ec5768b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec5768b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 -- 172.21.15.62:0/1257838345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5735a2b0 con 0x55ec5768e810 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb754bc5640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb740011d80 tx=0x7fb740015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277dee0ddb386435 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.524+0000 7fb738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740019040 con 0x55ec5768e810 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb740010b40 con 0x55ec5768e810 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb740010870 con 0x55ec5768e810 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb74000f960 con 0x55ec5768e810 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec5769cfe0 con 0x55ec5768e810 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb740010b40 con 0x55ec5768e810 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb74002c060 con 0x55ec5768e810 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 20 client.15270 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5769f9e0 0x55ec576a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 4 client.15270 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.525+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec575f5c80 con 0x55ec5769f9e0 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.526+0000 7fb7553c6640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5769f9e0 0x55ec576a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.526+0000 7fb738ff9640 10 client.15270 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.526+0000 7fb7553c6640 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5769f9e0 0x55ec576a0e50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e3ed314c466015 server_cookie=5cec735e5b641abb in_seq=0 out_seq=0 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb738ff9640 1 -- 172.21.15.62:0/1257838345 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec575f5c80 con 0x55ec5769f9e0 2024-08-16T19:10:21.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb738ff9640 10 client.15270 handle_command_reply: tid=0 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 client.15270 shutdown 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 msgr2=0x55ec5768c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb740011d80 tx=0x7fb740015040 comp rx=0 tx=0).stop 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 shutdown_connections 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5769f9e0 0x55ec576a0e50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec5768c160 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec5768bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 --2- 172.21.15.62:0/1257838345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec5768b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb738ff9640 0 client.15270 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 >> 172.21.15.62:0/1257838345 conn(0x55ec5767fbc0 msgr2=0x55ec576858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 shutdown_connections 2024-08-16T19:10:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 1 -- 172.21.15.62:0/1257838345 wait complete. 2024-08-16T19:10:21.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.527+0000 7fb759a28740 20 client.15270 trim_cache size 0 max 16384 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.545+0000 7fb759a28740 1 Processor -- start 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.545+0000 7fb759a28740 1 -- start start 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec5768aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ec5769f190 con 0x55ec57683f50 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ec5769f860 con 0x55ec57683630 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb759a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ec5769ff30 con 0x55ec5768e810 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34942/0 (socket says 172.21.15.62:34942) 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 -- 172.21.15.62:0/354261266 learned_addr learned my addr 172.21.15.62:0/354261266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb754bc5640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec5768aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb754bc5640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 -- 172.21.15.62:0/354261266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 msgr2=0x55ec5768aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec5768aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 -- 172.21.15.62:0/354261266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 msgr2=0x55ec576a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 -- 172.21.15.62:0/354261266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5767b130 con 0x55ec57683f50 2024-08-16T19:10:21.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb754bc5640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.546+0000 7fb7553c6640 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb74c02a940 tx=0x7fb74c031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176d95ffd979b60 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb73affd640 1 -- 172.21.15.62:0/354261266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c04d070 con 0x55ec57683f50 2024-08-16T19:10:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb73affd640 1 -- 172.21.15.62:0/354261266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c005500 con 0x55ec57683f50 2024-08-16T19:10:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb73affd640 1 -- 172.21.15.62:0/354261266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c035040 con 0x55ec57683f50 2024-08-16T19:10:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 -- 172.21.15.62:0/354261266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 msgr2=0x55ec57686090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb74c02a940 tx=0x7fb74c031b20 comp rx=0 tx=0).stop 2024-08-16T19:10:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 -- 172.21.15.62:0/354261266 shutdown_connections 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec5768e810 0x55ec5768aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec57683f50 0x55ec57686090 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 --2- 172.21.15.62:0/354261266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 -- 172.21.15.62:0/354261266 >> 172.21.15.62:0/354261266 conn(0x55ec5767fbc0 msgr2=0x55ec5768ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 -- 172.21.15.62:0/354261266 shutdown_connections 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.547+0000 7fb759a28740 1 -- 172.21.15.62:0/354261266 wait complete. 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 -- start start 2024-08-16T19:10:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 Processor -- start 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec57688630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ec576a0cd0 con 0x55ec5768e810 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ec576a0600 con 0x55ec57683630 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ec576a13a0 con 0x55ec57683f50 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb759a28740 10 client.0 _ll_register_callbacks cb 0x7fb740013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb754bc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34944/0 (socket says 172.21.15.62:34944) 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb754bc5640 1 -- 172.21.15.62:0/3132482557 learned_addr learned my addr 172.21.15.62:0/3132482557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.548+0000 7fb7553c6640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec57688630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 -- 172.21.15.62:0/3132482557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 msgr2=0x55ec57688630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec57688630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 -- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 msgr2=0x55ec576880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 -- 172.21.15.62:0/3132482557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec57688d20 con 0x55ec5768e810 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb754bc5640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb740001eb0 tx=0x7fb74000f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26411ae35f949927 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740019070 con 0x55ec5768e810 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb74000bd60 con 0x55ec5768e810 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74c005500 con 0x55ec5768e810 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb74000fb00 con 0x55ec5768e810 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb759a28740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-08-16T19:10:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.549+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ec57709700 con 0x55ec5768e810 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c005500 con 0x55ec5768e810 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fb74002c040 con 0x55ec5768e810 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 20 client.15276 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5770e140 0x55ec5770ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 4 client.15276 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "status"}) -- 0x55ec575f0190 con 0x55ec5770e140 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.550+0000 7fb7553c6640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5770e140 0x55ec5770ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb738ff9640 10 client.15276 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb7553c6640 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5770e140 0x55ec5770ce50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2523dd359690fb5c server_cookie=5224d0db1585cb29 in_seq=0 out_seq=0 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb738ff9640 1 -- 172.21.15.62:0/3132482557 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ec575f0190 con 0x55ec5770e140 2024-08-16T19:10:21.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb738ff9640 10 client.15276 handle_command_reply: tid=0 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 client.15276 shutdown 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 msgr2=0x55ec577080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb740001eb0 tx=0x7fb74000f7c0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 shutdown_connections 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ec5770e140 0x55ec5770ce50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec5768e810 0x55ec577080e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ec57683f50 0x55ec57688630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 --2- 172.21.15.62:0/3132482557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec57683630 0x55ec576880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 >> 172.21.15.62:0/3132482557 conn(0x55ec5767fbc0 msgr2=0x55ec57699fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb738ff9640 0 client.15276 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 shutdown_connections 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 1 -- 172.21.15.62:0/3132482557 wait complete. 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.551+0000 7fb759a28740 20 client.15276 trim_cache size 0 max 16384 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24476, 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:21.558 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 135.97374740199999, 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 27, 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 92, 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 88, 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 147.01192035899999 2024-08-16T19:10:21.559 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:21.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 msgr2=0x7fb73c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 0x7fb73c0a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7480cc130 tx=0x7fb748004a40 comp rx=0 tx=0).stop 2024-08-16T19:10:21.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 msgr2=0x7fb75019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb74800b970 tx=0x7fb74800be30 comp rx=0 tx=0).stop 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 shutdown_connections 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7501078b0 0x7fb75019cbd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb73c0a2b80 0x7fb73c0a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb750102250 0x7fb750198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 --2- 172.21.15.62:0/2125416933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7501018b0 0x7fb750198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 >> 172.21.15.62:0/2125416933 conn(0x7fb7500fd5f0 msgr2=0x7fb7500fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 shutdown_connections 2024-08-16T19:10:21.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:21.552+0000 7fb75762a640 1 -- 172.21.15.62:0/2125416933 wait complete. 2024-08-16T19:10:22.336 INFO:teuthology.run_tasks:Running task sequential... 2024-08-16T19:10:22.345 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-16T19:10:22.345 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-16T19:10:22.346 INFO:teuthology.task.print:Enabling mgr modules 2024-08-16T19:10:22.346 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-08-16T19:10:22.347 INFO:teuthology.task.exec:Executing custom commands... 2024-08-16T19:10:22.347 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:10:22.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-08-16T19:10:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.453+0000 7f1882046640 1 -- 172.21.15.62:0/5589471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 msgr2=0x7f187c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.453+0000 7f1882046640 1 --2- 172.21.15.62:0/5589471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c108bd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1874009ca0 tx=0x7f187402a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 -- 172.21.15.62:0/5589471 shutdown_connections 2024-08-16T19:10:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 --2- 172.21.15.62:0/5589471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c108bd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 --2- 172.21.15.62:0/5589471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c101980 0x7f187c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 --2- 172.21.15.62:0/5589471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c100790 0x7f187c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 -- 172.21.15.62:0/5589471 >> 172.21.15.62:0/5589471 conn(0x7f187c0fbf10 msgr2=0x7f187c0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 -- 172.21.15.62:0/5589471 shutdown_connections 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 -- 172.21.15.62:0/5589471 wait complete. 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 Processor -- start 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 -- start start 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.454+0000 7f1882046640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c100790 0x7f187c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c101980 0x7f187c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10c410 con 0x7f187c102340 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10c290 con 0x7f187c100790 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10c590 con 0x7f187c101980 2024-08-16T19:10:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34964/0 (socket says 172.21.15.62:34964) 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 -- 172.21.15.62:0/553681817 learned_addr learned my addr 172.21.15.62:0/553681817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187affd640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c101980 0x7f187c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187b7fe640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c100790 0x7f187c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 -- 172.21.15.62:0/553681817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c101980 msgr2=0x7f187c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c101980 0x7f187c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 -- 172.21.15.62:0/553681817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c100790 msgr2=0x7f187c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c100790 0x7f187c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 -- 172.21.15.62:0/553681817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c19f6f0 con 0x7f187c102340 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f187bfff640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1874031ae0 tx=0x7f1874031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874003c30 con 0x7f187c102340 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874003dd0 con 0x7f187c102340 2024-08-16T19:10:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c19f980 con 0x7f187c102340 2024-08-16T19:10:22.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187402fcb0 con 0x7f187c102340 2024-08-16T19:10:22.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.455+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c19fed0 con 0x7f187c102340 2024-08-16T19:10:22.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.457+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c1a00a0 con 0x7f187c102340 2024-08-16T19:10:22.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.458+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1874004050 con 0x7f187c102340 2024-08-16T19:10:22.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.459+0000 7f1878ff9640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 0x7f18640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:22.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.459+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f18740f08b0 con 0x7f187c102340 2024-08-16T19:10:22.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.459+0000 7f187b7fe640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 0x7f18640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:22.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.459+0000 7f187b7fe640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 0x7f18640a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f187c1017e0 tx=0x7f186c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:22.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.462+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f18740bd460 con 0x7f187c102340 2024-08-16T19:10:22.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:22.706+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f187c1a0430 con 0x7f187c102340 2024-08-16T19:10:23.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.002+0000 7f1878ff9640 1 -- 172.21.15.62:0/553681817 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f18740bcd60 con 0x7f187c102340 2024-08-16T19:10:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 msgr2=0x7f18640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 0x7f18640a50f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f187c1017e0 tx=0x7f186c005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 msgr2=0x7f187c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1874031ae0 tx=0x7f1874031ee0 comp rx=0 tx=0).stop 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 shutdown_connections 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f187c102340 0x7f187c19f080 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f187c101980 0x7f187c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f18640a2c30 0x7f18640a50f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:23.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 --2- 172.21.15.62:0/553681817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c100790 0x7f187c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 >> 172.21.15.62:0/553681817 conn(0x7f187c0fbf10 msgr2=0x7f187c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 shutdown_connections 2024-08-16T19:10:23.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.005+0000 7f1882046640 1 -- 172.21.15.62:0/553681817 wait complete. 2024-08-16T19:10:23.022 INFO:teuthology.run_tasks:Running task kclient... 2024-08-16T19:10:23.038 INFO:tasks.kclient:Mounting kernel clients... 2024-08-16T19:10:23.038 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-16T19:10:23.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:23.038 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-08-16T19:10:23.056 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-16T19:10:23.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:23.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-16T19:10:23.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:23.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-08-16T19:10:23.238 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-08-16T19:10:23.238 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-16T19:10:23.238 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:23.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-16T19:10:23.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:22 smithi062 ceph-mon[32993]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:23.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:22 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-16T19:10:23.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setuser ceph since I am not root 2024-08-16T19:10:23.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ignoring --setgroup ceph since I am not root 2024-08-16T19:10:23.247 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:23.169+0000 7fee53dc3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:10:23.305 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:22 smithi161 ceph-mon[37576]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:23.306 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:22 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-16T19:10:23.306 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: ignoring --setuser ceph since I am not root 2024-08-16T19:10:23.306 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: ignoring --setgroup ceph since I am not root 2024-08-16T19:10:23.306 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-mgr[39390]: -- 172.21.15.161:0/2165386496 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55771f43f0e0 con 0x55771f41d000 2024-08-16T19:10:23.306 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:23.158+0000 7fd26f586040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:22 smithi155 ceph-mon[37413]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:22 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: ignoring --setuser ceph since I am not root 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: ignoring --setgroup ceph since I am not root 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-mgr[38789]: -- 172.21.15.155:0/1550060756 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc6c0d90e0 con 0x55bc6c0b6800 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:23.158+0000 7f74a1c7d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-16T19:10:23.307 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:23.304+0000 7f74a1c7d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:23.510 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:23.579 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:23.370+0000 7f74a1c7d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:10:23.607 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:23.303+0000 7fd26f586040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:23.608 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:23.369+0000 7fd26f586040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:10:23.640 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:23.331+0000 7fee53dc3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:23.640 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:23.403+0000 7fee53dc3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-16T19:10:23.887 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:23.607+0000 7fd26f586040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:10:23.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:23.667+0000 7fd26f586040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:10:23.898 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:23.676+0000 7fee53dc3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:10:23.898 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:23.739+0000 7fee53dc3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:10:24.004 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:23.609+0000 7f74a1c7d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-16T19:10:24.004 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:23.668+0000 7f74a1c7d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-16T19:10:24.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 -- 172.21.15.62:0/836392851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1241022a0 msgr2=0x7fc1241026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:24.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/836392851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1241022a0 0x7fc1241026e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc114009900 tx=0x7fc11402a480 comp rx=0 tx=0).stop 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 -- 172.21.15.62:0/836392851 shutdown_connections 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/836392851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc124102c20 0x7fc12410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/836392851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1241022a0 0x7fc1241026e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/836392851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124100b50 0x7fc124100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 -- 172.21.15.62:0/836392851 >> 172.21.15.62:0/836392851 conn(0x7fc1240fc500 msgr2=0x7fc1240fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 -- 172.21.15.62:0/836392851 shutdown_connections 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.982+0000 7fc12b4f4640 1 -- 172.21.15.62:0/836392851 wait complete. 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 Processor -- start 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 -- start start 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 0x7fc1240fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:24.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124102c20 0x7fc1240fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc124110cb0 con 0x7fc124100b50 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc124110b30 con 0x7fc1241022a0 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc12b4f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc124110e30 con 0x7fc124102c20 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc129290640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc129290640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34998/0 (socket says 172.21.15.62:34998) 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc129290640 1 -- 172.21.15.62:0/2325731042 learned_addr learned my addr 172.21.15.62:0/2325731042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc128a8f640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 0x7fc1240fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.983+0000 7fc129a91640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124102c20 0x7fc1240fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124102c20 msgr2=0x7fc1240fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124102c20 0x7fc1240fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 msgr2=0x7fc1240fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 0x7fc1240fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 -- 172.21.15.62:0/2325731042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1241a66c0 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc128a8f640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 0x7fc1240fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc129290640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d770 tx=0x7fc12000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120017070 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120005500 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1241a69b0 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120004e00 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.984+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a6e70 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.986+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124100f30 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.986+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fc120004fa0 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.987+0000 7fc11a7fc640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 0x7fc1100a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.987+0000 7fc128a8f640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 msgr2=0x7fc1100a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.987+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fc1200cee20 con 0x7fc124100b50 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.987+0000 7fc128a8f640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 0x7fc1100a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:23.992+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc12009b380 con 0x7fc124100b50 2024-08-16T19:10:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.188+0000 7fc128a8f640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 msgr2=0x7fc1100a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.188+0000 7fc128a8f640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 0x7fc1100a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:10:24.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:24 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-16T19:10:24.189 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:24 smithi062 ceph-mon[32993]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-16T19:10:24.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.210+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc124109dd0 con 0x7fc124100b50 2024-08-16T19:10:24.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.210+0000 7fc11a7fc640 1 -- 172.21.15.62:0/2325731042 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7fc12009f680 con 0x7fc124100b50 2024-08-16T19:10:24.211 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 msgr2=0x7fc1100a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 0x7fc1100a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 msgr2=0x7fc1240fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d770 tx=0x7fc12000dc30 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 shutdown_connections 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc124102c20 0x7fc1240fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fc1100a2bd0 0x7fc1100a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1241022a0 0x7fc1240fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 --2- 172.21.15.62:0/2325731042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc124100b50 0x7fc1240fc9b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 >> 172.21.15.62:0/2325731042 conn(0x7fc1240fc500 msgr2=0x7fc124069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 shutdown_connections 2024-08-16T19:10:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:24.212+0000 7fc12b4f4640 1 -- 172.21.15.62:0/2325731042 wait complete. 2024-08-16T19:10:24.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-16T19:10:24.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-mon[37413]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-16T19:10:24.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/553681817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-16T19:10:24.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-mon[37576]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-16T19:10:24.871 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:24.747+0000 7fee53dc3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:10:24.887 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:24.587+0000 7fd26f586040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:10:24.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:24.700+0000 7fd26f586040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:24.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:24.757+0000 7fd26f586040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-16T19:10:24.909 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-16T19:10:24.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:24.910 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:10:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42174sec preferred_lft 42174sec 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-08-16T19:10:24.968 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-08-16T19:10:24.968 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-16T19:10:24.968 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-16T19:10:24.969 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:10:24.974 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:24.595+0000 7f74a1c7d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-16T19:10:24.974 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:24.708+0000 7f74a1c7d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:24.974 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:24.765+0000 7f74a1c7d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:10:24.974 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:24.973+0000 7f74a1c7d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:10:25.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:25.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:25.113 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-16T19:10:25.137 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:24.870+0000 7fee53dc3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:25.137 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:24.933+0000 7fee53dc3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-16T19:10:25.137 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2325731042' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:10:25.139 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-08-16T19:10:25.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:25.141 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-08-16T19:10:25.198 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-08-16T19:10:25.198 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-16T19:10:25.198 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-16T19:10:25.199 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-16T19:10:25.199 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-16T19:10:25.199 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:10:25.237 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2325731042' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:10:25.237 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:24.967+0000 7fd26f586040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:10:25.238 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:25.028+0000 7fd26f586040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:10:25.238 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:25.094+0000 7fd26f586040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:10:25.238 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:25.237+0000 7fd26f586040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2325731042' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-16T19:10:25.243 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:25.035+0000 7f74a1c7d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:10:25.243 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:25.100+0000 7f74a1c7d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:10:25.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:25.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:25.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:25.331 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-08-16T19:10:25.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:25.347 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-08-16T19:10:25.403 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:10:25.403 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:10:25.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-16T19:10:25.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-16T19:10:25.404 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:10:25.463 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:25.168+0000 7fee53dc3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-16T19:10:25.463 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:25.234+0000 7fee53dc3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-16T19:10:25.463 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:25.306+0000 7fee53dc3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-16T19:10:25.463 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:25.462+0000 7fee53dc3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:25.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:25.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:25.525 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-16T19:10:25.525 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:10:25.525 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:10:25.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-16T19:10:25.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-16T19:10:25.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-16T19:10:25.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-16T19:10:25.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-16T19:10:25.526 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:10:25.578 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:25.242+0000 7f74a1c7d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-16T19:10:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:25.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:25.686 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-16T19:10:25.686 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-08-16T19:10:25.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-08-16T19:10:25.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-08-16T19:10:25.686 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-08-16T19:10:25.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:25.768 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-16T19:10:25.768 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:25.784 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-16T19:10:25.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-16T19:10:25.785 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:25.840 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-16T19:10:25.840 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-16T19:10:25.840 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/5dce479e-2db7-4669-b90f-338aacf1c0d8 2024-08-16T19:10:25.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:25.841 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/5dce479e-2db7-4669-b90f-338aacf1c0d8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-08-16T19:10:25.926 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-08-16T19:10:25.926 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-08-16T19:10:25.961 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:25.649+0000 7fd26f586040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:10:25.969 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:25.656+0000 7f74a1c7d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:10:25.996 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:25 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:25.924+0000 7fee53dc3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-16T19:10:26.082 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-08-16T19:10:26.082 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-08-16T19:10:26.082 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.155:3300,172.21.15.161:3300" 2024-08-16T19:10:26.082 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@bd001ebc-5c01-11ef-bccf-c7b262605968.cephfs=/volumes/qa/sv_1/5dce479e-2db7-4669-b90f-338aacf1c0d8 2024-08-16T19:10:26.082 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.62:3300/172.21.15.155:3300/172.21.15.161:3300" will pass to kernel 2024-08-16T19:10:26.083 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-16T19:10:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-08-16T19:10:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-16T19:10:26.083 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-08-16T19:10:26.084 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-16T19:10:26.085 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-16T19:10:26.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:10:26.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:26.113 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-16T19:10:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-16T19:10:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:26.213 INFO:teuthology.orchestra.run.smithi062.stdout:client24982 2024-08-16T19:10:26.213 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:10:26.213 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-16T19:10:26.230 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-16T19:10:26.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-16T19:10:26.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:26.328 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:25.968+0000 7f74a1c7d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:10:26.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.034+0000 7f74a1c7d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:10:26.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.184+0000 7f74a1c7d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:10:26.329 INFO:teuthology.orchestra.run.smithi062.stdout:client24982 2024-08-16T19:10:26.330 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-16T19:10:26.340 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:26.267+0000 7fee53dc3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:10:26.340 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:26.339+0000 7fee53dc3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:10:26.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-16T19:10:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:26.387 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:25.960+0000 7fd26f586040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-16T19:10:26.387 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.025+0000 7fd26f586040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-16T19:10:26.388 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.174+0000 7fd26f586040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:10:26.389 INFO:teuthology.orchestra.run.smithi062.stdout:client24982 2024-08-16T19:10:26.389 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968.client24982/status') 2024-08-16T19:10:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-16T19:10:26.492 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.24982 (3)172.21.15.62:0/1510045059 2024-08-16T19:10:26.492 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-08-16T19:10:26.492 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-08-16T19:10:26.492 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-08-16T19:10:26.492 INFO:teuthology.orchestra.run.smithi062.stderr:71 bytes copied, 2.5228e-05 s, 2.8 MB/s 2024-08-16T19:10:26.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-16T19:10:26.500 INFO:teuthology.run_tasks:Running task exec... 2024-08-16T19:10:26.508 INFO:teuthology.task.exec:Executing custom commands... 2024-08-16T19:10:26.509 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:10:26.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-16T19:10:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.616+0000 7fb2a555c640 1 -- 172.21.15.62:0/3850205499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0102a80 msgr2=0x7fb2a0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.616+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3850205499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0102a80 0x7fb2a0102f00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb294009900 tx=0x7fb29402a410 comp rx=0 tx=0).stop 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 -- 172.21.15.62:0/3850205499 shutdown_connections 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3850205499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3850205499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0102a80 0x7fb2a0102f00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3850205499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0101890 0x7fb2a0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 -- 172.21.15.62:0/3850205499 >> 172.21.15.62:0/3850205499 conn(0x7fb2a00fd010 msgr2=0x7fb2a00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 -- 172.21.15.62:0/3850205499 shutdown_connections 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 -- 172.21.15.62:0/3850205499 wait complete. 2024-08-16T19:10:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 Processor -- start 2024-08-16T19:10:26.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.617+0000 7fb2a555c640 1 -- start start 2024-08-16T19:10:26.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:26.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 0x7fb2a0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:26.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010d510 con 0x7fb2a0101890 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010d390 con 0x7fb2a0103440 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb2a555c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010d690 con 0x7fb2a0102a80 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 0x7fb2a0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35002/0 (socket says 172.21.15.62:35002) 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 -- 172.21.15.62:0/3378143719 learned_addr learned my addr 172.21.15.62:0/3378143719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29f7fe640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 msgr2=0x7fb2a0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 0x7fb2a0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 msgr2=0x7fb2a019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 -- 172.21.15.62:0/3378143719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a019f720 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29e7fc640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 0x7fb2a0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29f7fe640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb29effd640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e9c0 tx=0x7fb29000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.618+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29000cd80 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.619+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2900035f0 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.619+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a019fa10 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.619+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290010690 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.619+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a019ff30 con 0x7fb2a0101890 2024-08-16T19:10:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.620+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a0108bd0 con 0x7fb2a0101890 2024-08-16T19:10:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.621+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fb2900037a0 con 0x7fb2a0101890 2024-08-16T19:10:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.621+0000 7fb287fff640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 0x7fb2880a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.622+0000 7fb29e7fc640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 msgr2=0x7fb2880a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.622+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fb2900cf730 con 0x7fb2a0101890 2024-08-16T19:10:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.622+0000 7fb29e7fc640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 0x7fb2880a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:26.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.626+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb29009c2e0 con 0x7fb2a0101890 2024-08-16T19:10:26.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:26.501+0000 7fee53dc3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-16T19:10:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.823+0000 7fb29e7fc640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 msgr2=0x7fb2880a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:26.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.823+0000 7fb29e7fc640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 0x7fb2880a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:10:26.828 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.498+0000 7f74a1c7d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:10:26.829 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.560+0000 7f74a1c7d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:10:26.829 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.744+0000 7f74a1c7d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:10:26.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.867+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fb2a0062b20 con 0x7fb2a0101890 2024-08-16T19:10:26.875 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.484+0000 7fd26f586040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:10:26.875 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.545+0000 7fd26f586040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:10:26.875 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.728+0000 7fd26f586040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:10:26.875 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.874+0000 7fd26f586040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:10:26.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.919+0000 7fb287fff640 1 -- 172.21.15.62:0/3378143719 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fb29009bbe0 con 0x7fb2a0101890 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 msgr2=0x7fb2880a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 0x7fb2880a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 msgr2=0x7fb2a0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e9c0 tx=0x7fb29000ee80 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 shutdown_connections 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fb2880a2d00 0x7fb2880a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a0103440 0x7fb2a019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2a0102a80 0x7fb2a0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 --2- 172.21.15.62:0/3378143719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2a0101890 0x7fb2a0197b00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 >> 172.21.15.62:0/3378143719 conn(0x7fb2a00fd010 msgr2=0x7fb2a0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 shutdown_connections 2024-08-16T19:10:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:26.921+0000 7fb2a555c640 1 -- 172.21.15.62:0/3378143719 wait complete. 2024-08-16T19:10:26.934 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-16T19:10:26.944 INFO:teuthology.run_tasks:Running task exec... 2024-08-16T19:10:26.952 INFO:teuthology.task.exec:Executing custom commands... 2024-08-16T19:10:26.952 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:10:26.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-08-16T19:10:27.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 -- 172.21.15.62:0/2507003570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 msgr2=0x7fe890108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 --2- 172.21.15.62:0/2507003570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe890108bc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe888009ca0 tx=0x7fe88802a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 -- 172.21.15.62:0/2507003570 shutdown_connections 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 --2- 172.21.15.62:0/2507003570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe890108bc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 --2- 172.21.15.62:0/2507003570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890101970 0x7fe890101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 --2- 172.21.15.62:0/2507003570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890100780 0x7fe890100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 -- 172.21.15.62:0/2507003570 >> 172.21.15.62:0/2507003570 conn(0x7fe8900fbee0 msgr2=0x7fe8900fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 -- 172.21.15.62:0/2507003570 shutdown_connections 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.060+0000 7fe897a9e640 1 -- 172.21.15.62:0/2507003570 wait complete. 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 Processor -- start 2024-08-16T19:10:27.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 -- start start 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890100780 0x7fe89006a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890101970 0x7fe89019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe89010c430 con 0x7fe890102330 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe89010c2b0 con 0x7fe890100780 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe897a9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe89010c5b0 con 0x7fe890101970 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe89603b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe89603b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35026/0 (socket says 172.21.15.62:35026) 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe895039640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890101970 0x7fe89019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe89603b640 1 -- 172.21.15.62:0/3096618027 learned_addr learned my addr 172.21.15.62:0/3096618027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.061+0000 7fe89583a640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890100780 0x7fe89006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 -- 172.21.15.62:0/3096618027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890101970 msgr2=0x7fe89019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890101970 0x7fe89019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 -- 172.21.15.62:0/3096618027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890100780 msgr2=0x7fe89006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890100780 0x7fe89006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 -- 172.21.15.62:0/3096618027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89019f720 con 0x7fe890102330 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe89603b640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe888009430 tx=0x7fe8880319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888003b70 con 0x7fe890102330 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89019f950 con 0x7fe890102330 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe888003d10 con 0x7fe890102330 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88802fc50 con 0x7fe890102330 2024-08-16T19:10:27.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.062+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89019fea0 con 0x7fe890102330 2024-08-16T19:10:27.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.063+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe890107a40 con 0x7fe890102330 2024-08-16T19:10:27.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.065+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fe888004b40 con 0x7fe890102330 2024-08-16T19:10:27.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.066+0000 7fe886ffd640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 0x7fe87c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.066+0000 7fe89583a640 1 -- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 msgr2=0x7fe87c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:27.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.066+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7fe8880f0ab0 con 0x7fe890102330 2024-08-16T19:10:27.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.066+0000 7fe89583a640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 0x7fe87c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:27.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.069+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe8880bd660 con 0x7fe890102330 2024-08-16T19:10:27.121 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-16T19:10:27.122 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:26.852+0000 7fee53dc3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-16T19:10:27.122 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:26 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:26.917+0000 7fee53dc3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-16T19:10:27.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-16T19:10:27.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:26.940+0000 7fd26f586040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:10:27.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:27.024+0000 7fd26f586040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:27.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:27.083+0000 7fd26f586040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:10:27.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.257+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fe890062c30 con 0x7fe890102330 2024-08-16T19:10:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.263+0000 7fe886ffd640 1 -- 172.21.15.62:0/3096618027 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7fe8880bcf60 con 0x7fe890102330 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 msgr2=0x7fe87c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 0x7fe87c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 msgr2=0x7fe89019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe888009430 tx=0x7fe8880319f0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 shutdown_connections 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe890102330 0x7fe89019ee10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe890101970 0x7fe89019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7fe87c0a2c80 0x7fe87c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 --2- 172.21.15.62:0/3096618027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe890100780 0x7fe89006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 >> 172.21.15.62:0/3096618027 conn(0x7fe8900fbee0 msgr2=0x7fe890104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 shutdown_connections 2024-08-16T19:10:27.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.265+0000 7fe897a9e640 1 -- 172.21.15.62:0/3096618027 wait complete. 2024-08-16T19:10:27.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-08-16T19:10:27.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.889+0000 7f74a1c7d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:10:27.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:26.954+0000 7f74a1c7d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:10:27.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:27.036+0000 7f74a1c7d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:27.329 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:27.096+0000 7f74a1c7d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:10:27.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:26 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-08-16T19:10:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 -- 172.21.15.62:0/664577073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 msgr2=0x7f0da40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 --2- 172.21.15.62:0/664577073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da40fff40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009900 tx=0x7f0d9402a410 comp rx=0 tx=0).stop 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 -- 172.21.15.62:0/664577073 shutdown_connections 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 --2- 172.21.15.62:0/664577073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da41034d0 0x7f0da4109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 --2- 172.21.15.62:0/664577073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da4102b10 0x7f0da4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 --2- 172.21.15.62:0/664577073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da40fff40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 -- 172.21.15.62:0/664577073 >> 172.21.15.62:0/664577073 conn(0x7f0da40fb6a0 msgr2=0x7f0da40fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 -- 172.21.15.62:0/664577073 shutdown_connections 2024-08-16T19:10:27.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.385+0000 7f0da8992640 1 -- 172.21.15.62:0/664577073 wait complete. 2024-08-16T19:10:27.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 Processor -- start 2024-08-16T19:10:27.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 -- start start 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da4102b10 0x7f0da4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da41034d0 0x7f0da4101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0da410d510 con 0x7f0da40ffb40 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0da410d390 con 0x7f0da41034d0 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da8992640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0da410d690 con 0x7f0da4102b10 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35038/0 (socket says 172.21.15.62:35038) 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 -- 172.21.15.62:0/3365427320 learned_addr learned my addr 172.21.15.62:0/3365427320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da2d9d640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da41034d0 0x7f0da4101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da1d9b640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da4102b10 0x7f0da4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da4102b10 msgr2=0x7f0da4100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da4102b10 0x7f0da4100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da41034d0 msgr2=0x7f0da4101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da41034d0 0x7f0da4101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 -- 172.21.15.62:0/3365427320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da4101890 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.386+0000 7f0da259c640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0d94031610 tx=0x7f0d94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.387+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9403f030 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.387+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da41a1bc0 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.387+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d94003ad0 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.387+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94035070 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.387+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da41a2160 con 0x7f0da40ffb40 2024-08-16T19:10:27.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.388+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da4108c90 con 0x7f0da40ffb40 2024-08-16T19:10:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.389+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f0d94042030 con 0x7f0da40ffb40 2024-08-16T19:10:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.390+0000 7f0d9b7fe640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 0x7f0d900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.390+0000 7f0da1d9b640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 msgr2=0x7f0d900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.390+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f0d940efd30 con 0x7f0da40ffb40 2024-08-16T19:10:27.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.390+0000 7f0da1d9b640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 0x7f0d900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:27.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.397+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0d940bc290 con 0x7f0da40ffb40 2024-08-16T19:10:27.447 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:27.121+0000 7fee53dc3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-16T19:10:27.447 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:27.281+0000 7fee53dc3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-16T19:10:27.447 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:27.352+0000 7fee53dc3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-16T19:10:27.447 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:27.446+0000 7fee53dc3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-16T19:10:27.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.584+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0da402d5e0 con 0x7f0da40ffb40 2024-08-16T19:10:27.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.590+0000 7f0d9b7fe640 1 -- 172.21.15.62:0/3365427320 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f0d940dae20 con 0x7f0da40ffb40 2024-08-16T19:10:27.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.591+0000 7f0da1d9b640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 msgr2=0x7f0d900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:27.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.591+0000 7f0da1d9b640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 0x7f0d900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:10:27.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 msgr2=0x7f0d900a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:27.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 0x7f0d900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 msgr2=0x7f0da4102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0d94031610 tx=0x7f0d94004270 comp rx=0 tx=0).stop 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 shutdown_connections 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f0d900a2c80 0x7f0d900a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0da41034d0 0x7f0da4101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0da4102b10 0x7f0da4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 --2- 172.21.15.62:0/3365427320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0da40ffb40 0x7f0da4102480 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 >> 172.21.15.62:0/3365427320 conn(0x7f0da40fb6a0 msgr2=0x7f0da4105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 shutdown_connections 2024-08-16T19:10:27.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.592+0000 7f0da8992640 1 -- 172.21.15.62:0/3365427320 wait complete. 2024-08-16T19:10:27.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-08-16T19:10:27.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 -- 172.21.15.62:0/2926402453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844102320 msgr2=0x7f7844108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 --2- 172.21.15.62:0/2926402453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844102320 0x7f7844108bb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ca0 tx=0x7f783c02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 -- 172.21.15.62:0/2926402453 shutdown_connections 2024-08-16T19:10:27.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 --2- 172.21.15.62:0/2926402453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844102320 0x7f7844108bb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 --2- 172.21.15.62:0/2926402453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 --2- 172.21.15.62:0/2926402453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844100770 0x7f7844100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 -- 172.21.15.62:0/2926402453 >> 172.21.15.62:0/2926402453 conn(0x7f78440fbf10 msgr2=0x7f78440fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 -- 172.21.15.62:0/2926402453 shutdown_connections 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.712+0000 7f784a924640 1 -- 172.21.15.62:0/2926402453 wait complete. 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 Processor -- start 2024-08-16T19:10:27.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 -- start start 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 0x7f7844103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f784410c3f0 con 0x7f7844100770 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f784410c270 con 0x7f7844102320 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f784a924640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f784410c570 con 0x7f7844101960 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f7843fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f7843fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35066/0 (socket says 172.21.15.62:35066) 2024-08-16T19:10:27.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f7843fff640 1 -- 172.21.15.62:0/965287068 learned_addr learned my addr 172.21.15.62:0/965287068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.713+0000 7f78437fe640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7848ec1640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 0x7f7844103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 msgr2=0x7f7844102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 msgr2=0x7f7844103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 0x7f7844103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 -- 172.21.15.62:0/965287068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7844103b10 con 0x7f7844100770 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f78437fe640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7848ec1640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 0x7f7844103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f7843fff640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9c0 tx=0x7f783400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400cd80 con 0x7f7844100770 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78340035f0 con 0x7f7844100770 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78441a1bd0 con 0x7f7844100770 2024-08-16T19:10:27.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834010690 con 0x7f7844100770 2024-08-16T19:10:27.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.714+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a2120 con 0x7f7844100770 2024-08-16T19:10:27.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.716+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78441a22f0 con 0x7f7844100770 2024-08-16T19:10:27.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.716+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f78340037a0 con 0x7f7844100770 2024-08-16T19:10:27.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.717+0000 7f78417fa640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 0x7f782c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:27.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.717+0000 7f78437fe640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 msgr2=0x7f782c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:27.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.717+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f78340cf940 con 0x7f7844100770 2024-08-16T19:10:27.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.717+0000 7f78437fe640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 0x7f782c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:27.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.721+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f783409c4f0 con 0x7f7844100770 2024-08-16T19:10:27.746 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:27.510+0000 7fee53dc3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-16T19:10:27.887 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:27.621+0000 7fd26f586040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:10:27.888 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z[39386]: 2024-08-16T19:10:27.783+0000 7fd26f586040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:10:27.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.918+0000 7f78437fe640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 msgr2=0x7f782c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:27.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.918+0000 7f78437fe640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 0x7f782c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:10:27.929 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:27.634+0000 7f74a1c7d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:10:27.929 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y[38785]: 2024-08-16T19:10:27.793+0000 7f74a1c7d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:10:27.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.948+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f78441a2ad0 con 0x7f7844100770 2024-08-16T19:10:27.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.952+0000 7f78417fa640 1 -- 172.21.15.62:0/965287068 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f783409bdf0 con 0x7f7844100770 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 msgr2=0x7f782c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 0x7f782c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 msgr2=0x7f7844104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9c0 tx=0x7f783400ee80 comp rx=0 tx=0).stop 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 shutdown_connections 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f782c0a2c80 0x7f782c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7844102320 0x7f7844103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7844101960 0x7f7844102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 --2- 172.21.15.62:0/965287068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7844100770 0x7f7844104700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:27.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 >> 172.21.15.62:0/965287068 conn(0x7f78440fbf10 msgr2=0x7f78440fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:27.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 shutdown_connections 2024-08-16T19:10:27.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:27.954+0000 7f784a924640 1 -- 172.21.15.62:0/965287068 wait complete. 2024-08-16T19:10:27.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-16T19:10:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.073+0000 7f1fea26d640 1 -- 172.21.15.62:0/3229831707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 msgr2=0x7f1fe4108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.073+0000 7f1fea26d640 1 --2- 172.21.15.62:0/3229831707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe4108bb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009ca0 tx=0x7f1fdc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 -- 172.21.15.62:0/3229831707 shutdown_connections 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 --2- 172.21.15.62:0/3229831707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe4108bb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 --2- 172.21.15.62:0/3229831707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe4101960 0x7f1fe4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 --2- 172.21.15.62:0/3229831707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 0x7f1fe4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 -- 172.21.15.62:0/3229831707 >> 172.21.15.62:0/3229831707 conn(0x7f1fe40fbf10 msgr2=0x7f1fe40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 -- 172.21.15.62:0/3229831707 shutdown_connections 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 -- 172.21.15.62:0/3229831707 wait complete. 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 Processor -- start 2024-08-16T19:10:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.074+0000 7f1fea26d640 1 -- start start 2024-08-16T19:10:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 0x7f1fe4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe4101960 0x7f1fe41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410c3f0 con 0x7f1fe4102320 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410c270 con 0x7f1fe4101960 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fea26d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410c570 con 0x7f1fe4100770 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35076/0 (socket says 172.21.15.62:35076) 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 -- 172.21.15.62:0/578509175 learned_addr learned my addr 172.21.15.62:0/578509175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 0x7f1fe4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe37fe640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe4101960 0x7f1fe41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 msgr2=0x7f1fe4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 0x7f1fe4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe4101960 msgr2=0x7f1fe41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe4101960 0x7f1fe41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe419f710 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fe4100770 0x7f1fe4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.075+0000 7f1fe880a640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe419f0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc031ae0 tx=0x7f1fdc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.076+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc003b70 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.076+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc003d10 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.076+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe419f9a0 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.076+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc02fc40 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.076+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe419ff40 con 0x7f1fe4102320 2024-08-16T19:10:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.077+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe41a0170 con 0x7f1fe4102320 2024-08-16T19:10:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.078+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f1fdc004050 con 0x7f1fe4102320 2024-08-16T19:10:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.079+0000 7f1fe17fa640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 0x7f1fcc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.079+0000 7f1fe3fff640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 msgr2=0x7f1fcc0a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.079+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9179+0+0 (secure 0 0 0) 0x7f1fdc0f06b0 con 0x7f1fe4102320 2024-08-16T19:10:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.079+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 0x7f1fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-16T19:10:28.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.082+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1fdc0bd260 con 0x7f1fe4102320 2024-08-16T19:10:28.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.130+0000 7fee53dc3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-16T19:10:28.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3096618027' entity='client.admin' 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3365427320' entity='client.admin' 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: Standby manager daemon z restarted 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: Standby manager daemon z started 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: Standby manager daemon y restarted 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: Standby manager daemon y started 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:10:28.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:10:28.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:27 smithi062 ceph-mon[32993]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.280+0000 7f1fe3fff640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 msgr2=0x7f1fcc0a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.62:6800/727495888 2024-08-16T19:10:28.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.280+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 0x7f1fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-16T19:10:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.294+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1fe41a0340 con 0x7f1fcc0a2d00 2024-08-16T19:10:28.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.313+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f1fdc0b91d0 con 0x7f1fe4102320 2024-08-16T19:10:28.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.314+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 msgr2=0x7f1fcc0a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-16T19:10:28.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:28.314+0000 7f1fe17fa640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/727495888,v1:172.21.15.62:6801/727495888] conn(0x7f1fcc0a2d00 0x7f1fcc0a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3096618027' entity='client.admin' 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3365427320' entity='client.admin' 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: Standby manager daemon z restarted 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: Standby manager daemon z started 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: Standby manager daemon y restarted 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: Standby manager daemon y started 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:10:28.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:27 smithi155 ceph-mon[37413]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3378143719' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3096618027' entity='client.admin' 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3365427320' entity='client.admin' 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: Standby manager daemon z restarted 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: Standby manager daemon z started 2024-08-16T19:10:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.161:0/925137701' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: Standby manager daemon y restarted 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: Standby manager daemon y started 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-16T19:10:28.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:27 smithi161 ceph-mon[37576]: from='mgr.? 172.21.15.155:0/1390706640' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-16T19:10:28.558 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.299+0000 7fee53dc3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.557+0000 7fedbdf92640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.557+0000 7fedbdf92640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.557+0000 7fedbdf92640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.557+0000 7fedbdf92640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.557+0000 7fedbdf92640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.559+0000 7fedba78b640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.559+0000 7fedba78b640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.559+0000 7fedba78b640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.818 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.559+0000 7fedba78b640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:28.819 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:28.559+0000 7fedba78b640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:10:29.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/965287068' entity='client.admin' 2024-08-16T19:10:29.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: Active manager daemon x restarted 2024-08-16T19:10:29.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: Activating manager daemon x 2024-08-16T19:10:29.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: osdmap e93: 12 total, 12 up, 12 in 2024-08-16T19:10:29.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: mgrmap e20: x(active, starting, since 0.0134656s), standbys: y, z 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:10:29.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:10:29.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: Manager daemon x is now available 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:10:29.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:28 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:10:29.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/965287068' entity='client.admin' 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: Active manager daemon x restarted 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: Activating manager daemon x 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: osdmap e93: 12 total, 12 up, 12 in 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: mgrmap e20: x(active, starting, since 0.0134656s), standbys: y, z 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:10:29.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:10:29.214 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:10:29.215 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:10:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: Manager daemon x is now available 2024-08-16T19:10:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:10:29.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:28 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:10:29.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.317+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mon.0 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f1fdc004340 con 0x7f1fe4102320 2024-08-16T19:10:29.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.318+0000 7f1fe17fa640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fcc051930 0x7f1fcc051d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.318+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1fcc00c450 con 0x7f1fcc051930 2024-08-16T19:10:29.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.318+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fcc051930 0x7f1fcc051d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.325+0000 7f1fe3fff640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fcc051930 0x7f1fcc051d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fe41017c0 tx=0x7f1fd4005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/965287068' entity='client.admin' 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: Active manager daemon x restarted 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: Activating manager daemon x 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: osdmap e93: 12 total, 12 up, 12 in 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: mgrmap e20: x(active, starting, since 0.0134656s), standbys: y, z 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-16T19:10:29.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:10:29.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-16T19:10:29.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: Manager daemon x is now available 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-16T19:10:29.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:28 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-16T19:10:29.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.392+0000 7f1fe17fa640 1 -- 172.21.15.62:0/578509175 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1fcc00c450 con 0x7f1fcc051930 2024-08-16T19:10:29.392 INFO:teuthology.orchestra.run.smithi062.stdout:Schedule set for path / 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fcc051930 msgr2=0x7f1fcc051d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fcc051930 0x7f1fcc051d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fe41017c0 tx=0x7f1fd4005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 msgr2=0x7f1fe419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 --2- 172.21.15.62:0/578509175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe4102320 0x7f1fe419f0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc031ae0 tx=0x7f1fdc031ee0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fe880a640 1 -- 172.21.15.62:0/578509175 reap_dead start 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 shutdown_connections 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 >> 172.21.15.62:0/578509175 conn(0x7f1fe40fbf10 msgr2=0x7f1fe40fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 shutdown_connections 2024-08-16T19:10:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.394+0000 7f1fea26d640 1 -- 172.21.15.62:0/578509175 wait complete. 2024-08-16T19:10:29.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2574296540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 msgr2=0x7f6dd8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2574296540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8101de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009900 tx=0x7f6dcc02a410 comp rx=0 tx=0).stop 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2574296540 shutdown_connections 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2574296540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8102320 0x7f6dd8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2574296540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8101de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2574296540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8100770 0x7f6dd8100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2574296540 >> 172.21.15.62:0/2574296540 conn(0x7f6dd80fbf10 msgr2=0x7f6dd80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:29.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2574296540 shutdown_connections 2024-08-16T19:10:29.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.518+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2574296540 wait complete. 2024-08-16T19:10:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 Processor -- start 2024-08-16T19:10:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 -- start start 2024-08-16T19:10:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8100770 0x7f6dd8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8102320 0x7f6dd819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd810d540 con 0x7f6dd8101960 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd810d3c0 con 0x7f6dd8102320 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6ddcda8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd810d6c0 con 0x7f6dd8100770 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6dd5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6dd5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35234/0 (socket says 172.21.15.62:35234) 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6dd5d9b640 1 -- 172.21.15.62:0/2785724534 learned_addr learned my addr 172.21.15.62:0/2785724534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.519+0000 7f6dd659c640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8100770 0x7f6dd8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd6d9d640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8102320 0x7f6dd819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 -- 172.21.15.62:0/2785724534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8100770 msgr2=0x7f6dd8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8100770 0x7f6dd8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 -- 172.21.15.62:0/2785724534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8102320 msgr2=0x7f6dd819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8102320 0x7f6dd819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 -- 172.21.15.62:0/2785724534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd819f710 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dd5d9b640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009230 tx=0x7f6dcc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc03f030 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dcc003ad0 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd819f9a0 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc035070 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.520+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd819fe90 con 0x7f6dd8101960 2024-08-16T19:10:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.521+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd8107ab0 con 0x7f6dd8101960 2024-08-16T19:10:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.522+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f6dcc042030 con 0x7f6dd8101960 2024-08-16T19:10:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.523+0000 7f6dc77fe640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 0x7f6dc00a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.523+0000 7f6dd659c640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 0x7f6dc00a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.523+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f6dcc0ef9c0 con 0x7f6dd8101960 2024-08-16T19:10:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.524+0000 7f6dd659c640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 0x7f6dc00a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005fd0 tx=0x7f6dc800b410 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.526+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6dcc0bc110 con 0x7f6dd8101960 2024-08-16T19:10:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.741+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f6dd80624c0 con 0x7f6dc00a2a40 2024-08-16T19:10:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.801+0000 7f6dc77fe640 1 -- 172.21.15.62:0/2785724534 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f6dd80624c0 con 0x7f6dc00a2a40 2024-08-16T19:10:29.802 INFO:teuthology.orchestra.run.smithi062.stdout:Retention added to path / 2024-08-16T19:10:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 msgr2=0x7f6dc00a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 0x7f6dc00a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005fd0 tx=0x7f6dc800b410 comp rx=0 tx=0).stop 2024-08-16T19:10:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 msgr2=0x7f6dd8198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009230 tx=0x7f6dcc031b60 comp rx=0 tx=0).stop 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 shutdown_connections 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6dc00a2a40 0x7f6dc00a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd8102320 0x7f6dd819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd8101960 0x7f6dd8198030 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 --2- 172.21.15.62:0/2785724534 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dd8100770 0x7f6dd8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 >> 172.21.15.62:0/2785724534 conn(0x7f6dd80fbf10 msgr2=0x7f6dd8104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 shutdown_connections 2024-08-16T19:10:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.803+0000 7f6ddcda8640 1 -- 172.21.15.62:0/2785724534 wait complete. 2024-08-16T19:10:29.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-16T19:10:29.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- 172.21.15.62:0/530403767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 msgr2=0x7f58d41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 --2- 172.21.15.62:0/530403767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41099c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009ca0 tx=0x7f58cc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- 172.21.15.62:0/530403767 shutdown_connections 2024-08-16T19:10:29.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 --2- 172.21.15.62:0/530403767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41099c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 --2- 172.21.15.62:0/530403767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 0x7f58d4107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 --2- 172.21.15.62:0/530403767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 0x7f58d41046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- 172.21.15.62:0/530403767 >> 172.21.15.62:0/530403767 conn(0x7f58d40fbf10 msgr2=0x7f58d40fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- 172.21.15.62:0/530403767 shutdown_connections 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- 172.21.15.62:0/530403767 wait complete. 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 Processor -- start 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.923+0000 7f58dbd65640 1 -- start start 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 0x7f58d4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 0x7f58d419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410d580 con 0x7f58d4107550 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410d400 con 0x7f58d41022f0 2024-08-16T19:10:29.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410d700 con 0x7f58d4104c20 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35254/0 (socket says 172.21.15.62:35254) 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 -- 172.21.15.62:0/2031166899 learned_addr learned my addr 172.21.15.62:0/2031166899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58d9300640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 0x7f58d419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58d9b01640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 0x7f58d4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 -- 172.21.15.62:0/2031166899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 msgr2=0x7f58d419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 0x7f58d419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 -- 172.21.15.62:0/2031166899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 msgr2=0x7f58d4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 0x7f58d4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 -- 172.21.15.62:0/2031166899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d41a19b0 con 0x7f58d4107550 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58da302640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f58cc031ae0 tx=0x7f58cc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003b70 con 0x7f58d4107550 2024-08-16T19:10:29.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58cc003d10 con 0x7f58d4107550 2024-08-16T19:10:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.924+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d41a1c40 con 0x7f58d4107550 2024-08-16T19:10:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.925+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc02fc00 con 0x7f58d4107550 2024-08-16T19:10:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.925+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a2190 con 0x7f58d4107550 2024-08-16T19:10:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.926+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d4109040 con 0x7f58d4107550 2024-08-16T19:10:29.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.928+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f58cc004690 con 0x7f58d4107550 2024-08-16T19:10:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.928+0000 7f58c6ffd640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 0x7f58c00a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.928+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f58cc0f07d0 con 0x7f58d4107550 2024-08-16T19:10:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.928+0000 7f58d9b01640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 0x7f58c00a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:29.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.929+0000 7f58d9b01640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 0x7f58c00a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58d419b270 tx=0x7f58d0005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:29.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:29.933+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f58cc0bd570 con 0x7f58d4107550 2024-08-16T19:10:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.147+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f58d4062360 con 0x7f58c00a2ac0 2024-08-16T19:10:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.154+0000 7f58c6ffd640 1 -- 172.21.15.62:0/2031166899 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f58d4062360 con 0x7f58c00a2ac0 2024-08-16T19:10:30.154 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-16T00:00:00", "created": "2024-08-16T19:10:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-16T19:10:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 msgr2=0x7f58c00a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 0x7f58c00a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58d419b270 tx=0x7f58d0005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 msgr2=0x7f58d41a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f58cc031ae0 tx=0x7f58cc0319f0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 shutdown_connections 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f58c00a2ac0 0x7f58c00a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d4107550 0x7f58d41a1310 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58d4104c20 0x7f58d419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 --2- 172.21.15.62:0/2031166899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d41022f0 0x7f58d4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 >> 172.21.15.62:0/2031166899 conn(0x7f58d40fbf10 msgr2=0x7f58d4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 shutdown_connections 2024-08-16T19:10:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.156+0000 7f58dbd65640 1 -- 172.21.15.62:0/2031166899 wait complete. 2024-08-16T19:10:30.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 -- 172.21.15.62:0/2900513182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4102330 msgr2=0x7fbdd4108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 --2- 172.21.15.62:0/2900513182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4102330 0x7fbdd4108bc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009ca0 tx=0x7fbdcc02a8c0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 -- 172.21.15.62:0/2900513182 shutdown_connections 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 --2- 172.21.15.62:0/2900513182 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4102330 0x7fbdd4108bc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 --2- 172.21.15.62:0/2900513182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4101970 0x7fbdd4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 --2- 172.21.15.62:0/2900513182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4100780 0x7fbdd4100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 -- 172.21.15.62:0/2900513182 >> 172.21.15.62:0/2900513182 conn(0x7fbdd40fbee0 msgr2=0x7fbdd40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 -- 172.21.15.62:0/2900513182 shutdown_connections 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.290+0000 7fbddae46640 1 -- 172.21.15.62:0/2900513182 wait complete. 2024-08-16T19:10:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 Processor -- start 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 -- start start 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4101970 0x7fbdd4109740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4102330 0x7fbdd4109c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410d550 con 0x7fbdd4100780 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410d3d0 con 0x7fbdd4102330 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbddae46640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd410d6d0 con 0x7fbdd4101970 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbdd8be2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbdd8be2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35268/0 (socket says 172.21.15.62:35268) 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbdd8be2640 1 -- 172.21.15.62:0/320479903 learned_addr learned my addr 172.21.15.62:0/320479903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbdd93e3640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4102330 0x7fbdd4109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.291+0000 7fbdd3fff640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4101970 0x7fbdd4109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 -- 172.21.15.62:0/320479903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4101970 msgr2=0x7fbdd4109740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4101970 0x7fbdd4109740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 -- 172.21.15.62:0/320479903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4102330 msgr2=0x7fbdd4109c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4102330 0x7fbdd4109c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 -- 172.21.15.62:0/320479903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd410a510 con 0x7fbdd4100780 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd8be2640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e9c0 tx=0x7fbdc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800cd80 con 0x7fbdd4100780 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd41a1ba0 con 0x7fbdd4100780 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc80035f0 con 0x7fbdd4100780 2024-08-16T19:10:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8010690 con 0x7fbdd4100780 2024-08-16T19:10:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.292+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd41a20f0 con 0x7fbdd4100780 2024-08-16T19:10:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.294+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd4107aa0 con 0x7fbdd4100780 2024-08-16T19:10:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.295+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fbdc80037a0 con 0x7fbdd4100780 2024-08-16T19:10:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.296+0000 7fbdd1ffb640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 0x7fbdc40a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.296+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fbdc80cf420 con 0x7fbdd4100780 2024-08-16T19:10:30.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.298+0000 7fbdd3fff640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 0x7fbdc40a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:30.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.298+0000 7fbdd3fff640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 0x7fbdc40a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00095f0 tx=0x7fbdc0005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.301+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbdc809c290 con 0x7fbdd4100780 2024-08-16T19:10:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.327+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7fbdc8098200 con 0x7fbdd4100780 2024-08-16T19:10:30.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:30 smithi062 ceph-mon[32993]: mgrmap e21: x(active, since 1.02139s), standbys: y, z 2024-08-16T19:10:30.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:30 smithi062 ceph-mon[32993]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:30 smithi062 ceph-mon[32993]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:30.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:30 smithi062 ceph-mon[32993]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:30 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:30.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.554+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fbdd4062360 con 0x7fbdc40a2a40 2024-08-16T19:10:30.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.561+0000 7fbdd1ffb640 1 -- 172.21.15.62:0/320479903 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fbdd4062360 con 0x7fbdc40a2a40 2024-08-16T19:10:30.562 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-08-16T19:10:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 msgr2=0x7fbdc40a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 0x7fbdc40a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00095f0 tx=0x7fbdc0005f90 comp rx=0 tx=0).stop 2024-08-16T19:10:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 msgr2=0x7fbdd410b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:30.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800e9c0 tx=0x7fbdc800ee80 comp rx=0 tx=0).stop 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 shutdown_connections 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbdc40a2a40 0x7fbdc40a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd4102330 0x7fbdd4109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbdd4101970 0x7fbdd4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 --2- 172.21.15.62:0/320479903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbdd4100780 0x7fbdd410b100 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 >> 172.21.15.62:0/320479903 conn(0x7fbdd40fbee0 msgr2=0x7fbdd4104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 shutdown_connections 2024-08-16T19:10:30.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:30.564+0000 7fbddae46640 1 -- 172.21.15.62:0/320479903 wait complete. 2024-08-16T19:10:30.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-16T19:10:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:30 smithi155 ceph-mon[37413]: mgrmap e21: x(active, since 1.02139s), standbys: y, z 2024-08-16T19:10:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:30 smithi155 ceph-mon[37413]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:30 smithi155 ceph-mon[37413]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:30 smithi155 ceph-mon[37413]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:30 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:30.606 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-16T19:10:30.615 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-16T19:10:30.637 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-16T19:10:30.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:10:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:30 smithi161 ceph-mon[37576]: mgrmap e21: x(active, since 1.02139s), standbys: y, z 2024-08-16T19:10:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:30 smithi161 ceph-mon[37576]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:30 smithi161 ceph-mon[37576]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:30 smithi161 ceph-mon[37576]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:10:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:30 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:30.928 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:31.322 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:31.233+0000 7fed9d712640 -1 log_channel(cephadm) log [ERR] : [16/Aug/2024:19:10:31] ENGINE Error in HTTPServer.serve 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Traceback (most recent call last): 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._connections.run(self.expiration_interval) 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._run(expiration_interval) 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:10:31.323 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: s = self.context.wrap_socket( 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: return self.sslsocket_class._create( 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.do_handshake() 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self._sslobj.do_handshake() 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:10:31.324 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 -- 172.21.15.62:0/4177863763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b4102b00 msgr2=0x7f28b4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 --2- 172.21.15.62:0/4177863763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b4102b00 0x7f28b4102ee0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009900 tx=0x7f28b002a400 comp rx=0 tx=0).stop 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 -- 172.21.15.62:0/4177863763 shutdown_connections 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 --2- 172.21.15.62:0/4177863763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 0x7f28b410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 --2- 172.21.15.62:0/4177863763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b41034b0 0x7f28b41078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 --2- 172.21.15.62:0/4177863763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b4102b00 0x7f28b4102ee0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 -- 172.21.15.62:0/4177863763 >> 172.21.15.62:0/4177863763 conn(0x7f28b40fc500 msgr2=0x7f28b40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 -- 172.21.15.62:0/4177863763 shutdown_connections 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.434+0000 7f28bbda4640 1 -- 172.21.15.62:0/4177863763 wait complete. 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 Processor -- start 2024-08-16T19:10:31.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 -- start start 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 0x7f28b41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 0x7f28b41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 0x7f28b419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4111e00 con 0x7f28b4102b00 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4111c80 con 0x7f28b4108030 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28bbda4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4111f80 con 0x7f28b41034b0 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28b9b40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 0x7f28b41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28b9b40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 0x7f28b41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44290/0 (socket says 172.21.15.62:44290) 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28b9b40640 1 -- 172.21.15.62:0/1516440537 learned_addr learned my addr 172.21.15.62:0/1516440537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.435+0000 7f28b933f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 0x7f28b41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28ba341640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 0x7f28b419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:31.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 -- 172.21.15.62:0/1516440537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 msgr2=0x7f28b419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 0x7f28b419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 -- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 msgr2=0x7f28b41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 0x7f28b41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 -- 172.21.15.62:0/1516440537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b419d260 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28b933f640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 0x7f28b41987f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28b4069500 tx=0x7f28a400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a400cc40 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a400cde0 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b419d550 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40188e0 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.436+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a4e30 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.437+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b406a4e0 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.439+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f28a4018a80 con 0x7f28b41034b0 2024-08-16T19:10:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.439+0000 7f28aaffd640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 0x7f28a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.439+0000 7f28b9b40640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 0x7f28a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.439+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f28a40cfcc0 con 0x7f28b41034b0 2024-08-16T19:10:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.440+0000 7f28b9b40640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 0x7f28a00a5230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005770 tx=0x7f28b0005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.442+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f28a409bf00 con 0x7f28b41034b0 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: [16/Aug/2024:19:10:31] ENGINE Bus STARTING 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:31.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:31 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: [16/Aug/2024:19:10:31] ENGINE Bus STARTING 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:31 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:31.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.641+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28b4062360 con 0x7f28a00a2d70 2024-08-16T19:10:31.641 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:10:31.642 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: [16/Aug/2024:19:10:31] ENGINE Bus STARTING 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.643 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:10:31.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:31.644 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:31 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.650+0000 7f28aaffd640 1 -- 172.21.15.62:0/1516440537 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339436 (secure 0 0 0) 0x7f28b4062360 con 0x7f28a00a2d70 2024-08-16T19:10:31.655 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:31.657 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:10:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.654+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 msgr2=0x7f28a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.654+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 0x7f28a00a5230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005770 tx=0x7f28b0005a20 comp rx=0 tx=0).stop 2024-08-16T19:10:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.654+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 msgr2=0x7f28b41987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.654+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 0x7f28b41987f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f28b4069500 tx=0x7f28a400ee50 comp rx=0 tx=0).stop 2024-08-16T19:10:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 shutdown_connections 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f28a00a2d70 0x7f28a00a5230 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28b4108030 0x7f28b419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28b41034b0 0x7f28b41987f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 --2- 172.21.15.62:0/1516440537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28b4102b00 0x7f28b41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 >> 172.21.15.62:0/1516440537 conn(0x7f28b40fc500 msgr2=0x7f28b4105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 shutdown_connections 2024-08-16T19:10:31.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:31.655+0000 7f28bbda4640 1 -- 172.21.15.62:0/1516440537 wait complete. 2024-08-16T19:10:32.424 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-08-16T19:10:30.321455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193773,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1296,"num_read_kb":813,"num_write":568,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381876,"kb_used_data":15988,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559948,"statfs":{"total":1151940427776,"available":1151549386752,"internally_reserved":0,"allocated":16371712,"data_stored":9597098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989849"},"pg_stats":[{"pgid":"2.1f","version":"93'15","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.558869+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:10:28.558869+0000","last_peered":"2024-08-16T19:10:28.558869+0000","last_clean":"2024-08-16T19:10:28.558869+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:10:28.558869+0000","last_undegraded":"2024-08-16T19:10:28.558869+0000","last_fullsized":"2024-08-16T19:10:28.558869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406289+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:10:28.406289+0000","last_peered":"2024-08-16T19:10:28.406289+0000","last_clean":"2024-08-16T19:10:28.406289+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:10:28.406289+0000","last_undegraded":"2024-08-16T19:10:28.406289+0000","last_fullsized":"2024-08-16T19:10:28.406289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415552+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:10:28.415552+0000","last_peered":"2024-08-16T19:10:28.415552+0000","last_clean":"2024-08-16T19:10:28.415552+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:10:28.415552+0000","last_undegraded":"2024-08-16T19:10:28.415552+0000","last_fullsized":"2024-08-16T19:10:28.415552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406640+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:10:28.406640+0000","last_peered":"2024-08-16T19:10:28.406640+0000","last_clean":"2024-08-16T19:10:28.406640+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:10:28.406640+0000","last_undegraded":"2024-08-16T19:10:28.406640+0000","last_fullsized":"2024-08-16T19:10:28.406640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395855+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:10:28.395855+0000","last_peered":"2024-08-16T19:10:28.395855+0000","last_clean":"2024-08-16T19:10:28.395855+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:10:28.395855+0000","last_undegraded":"2024-08-16T19:10:28.395855+0000","last_fullsized":"2024-08-16T19:10:28.395855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527883+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:10:28.527883+0000","last_peered":"2024-08-16T19:10:28.527883+0000","last_clean":"2024-08-16T19:10:28.527883+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:10:28.527883+0000","last_undegraded":"2024-08-16T19:10:28.527883+0000","last_fullsized":"2024-08-16T19:10:28.527883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404475+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:10:28.404475+0000","last_peered":"2024-08-16T19:10:28.404475+0000","last_clean":"2024-08-16T19:10:28.404475+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:10:28.404475+0000","last_undegraded":"2024-08-16T19:10:28.404475+0000","last_fullsized":"2024-08-16T19:10:28.404475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415713+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:10:28.415713+0000","last_peered":"2024-08-16T19:10:28.415713+0000","last_clean":"2024-08-16T19:10:28.415713+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:10:28.415713+0000","last_undegraded":"2024-08-16T19:10:28.415713+0000","last_fullsized":"2024-08-16T19:10:28.415713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528080+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:10:28.528080+0000","last_peered":"2024-08-16T19:10:28.528080+0000","last_clean":"2024-08-16T19:10:28.528080+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:10:28.528080+0000","last_undegraded":"2024-08-16T19:10:28.528080+0000","last_fullsized":"2024-08-16T19:10:28.528080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404392+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:10:28.404392+0000","last_peered":"2024-08-16T19:10:28.404392+0000","last_clean":"2024-08-16T19:10:28.404392+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:10:28.404392+0000","last_undegraded":"2024-08-16T19:10:28.404392+0000","last_fullsized":"2024-08-16T19:10:28.404392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400420+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:10:28.400420+0000","last_peered":"2024-08-16T19:10:28.400420+0000","last_clean":"2024-08-16T19:10:28.400420+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:10:28.400420+0000","last_undegraded":"2024-08-16T19:10:28.400420+0000","last_fullsized":"2024-08-16T19:10:28.400420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527974+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:10:28.527974+0000","last_peered":"2024-08-16T19:10:28.527974+0000","last_clean":"2024-08-16T19:10:28.527974+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:10:28.527974+0000","last_undegraded":"2024-08-16T19:10:28.527974+0000","last_fullsized":"2024-08-16T19:10:28.527974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416184+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:10:28.416184+0000","last_peered":"2024-08-16T19:10:28.416184+0000","last_clean":"2024-08-16T19:10:28.416184+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:10:28.416184+0000","last_undegraded":"2024-08-16T19:10:28.416184+0000","last_fullsized":"2024-08-16T19:10:28.416184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404284+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:10:28.404284+0000","last_peered":"2024-08-16T19:10:28.404284+0000","last_clean":"2024-08-16T19:10:28.404284+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:10:28.404284+0000","last_undegraded":"2024-08-16T19:10:28.404284+0000","last_fullsized":"2024-08-16T19:10:28.404284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528162+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:10:28.528162+0000","last_peered":"2024-08-16T19:10:28.528162+0000","last_clean":"2024-08-16T19:10:28.528162+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:10:28.528162+0000","last_undegraded":"2024-08-16T19:10:28.528162+0000","last_fullsized":"2024-08-16T19:10:28.528162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404992+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:10:28.404992+0000","last_peered":"2024-08-16T19:10:28.404992+0000","last_clean":"2024-08-16T19:10:28.404992+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:10:28.404992+0000","last_undegraded":"2024-08-16T19:10:28.404992+0000","last_fullsized":"2024-08-16T19:10:28.404992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416087+0000","last_change":"2024-08-16T19:09:59.583331+0000","last_active":"2024-08-16T19:10:28.416087+0000","last_peered":"2024-08-16T19:10:28.416087+0000","last_clean":"2024-08-16T19:10:28.416087+0000","last_became_active":"2024-08-16T19:09:59.578673+0000","last_became_peered":"2024-08-16T19:09:59.578673+0000","last_unstale":"2024-08-16T19:10:28.416087+0000","last_undegraded":"2024-08-16T19:10:28.416087+0000","last_fullsized":"2024-08-16T19:10:28.416087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:00:56.841049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528119+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:10:28.528119+0000","last_peered":"2024-08-16T19:10:28.528119+0000","last_clean":"2024-08-16T19:10:28.528119+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:10:28.528119+0000","last_undegraded":"2024-08-16T19:10:28.528119+0000","last_fullsized":"2024-08-16T19:10:28.528119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415835+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:10:28.415835+0000","last_peered":"2024-08-16T19:10:28.415835+0000","last_clean":"2024-08-16T19:10:28.415835+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:10:28.415835+0000","last_undegraded":"2024-08-16T19:10:28.415835+0000","last_fullsized":"2024-08-16T19:10:28.415835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405227+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:10:28.405227+0000","last_peered":"2024-08-16T19:10:28.405227+0000","last_clean":"2024-08-16T19:10:28.405227+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:10:28.405227+0000","last_undegraded":"2024-08-16T19:10:28.405227+0000","last_fullsized":"2024-08-16T19:10:28.405227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400990+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:10:28.400990+0000","last_peered":"2024-08-16T19:10:28.400990+0000","last_clean":"2024-08-16T19:10:28.400990+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:10:28.400990+0000","last_undegraded":"2024-08-16T19:10:28.400990+0000","last_fullsized":"2024-08-16T19:10:28.400990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404046+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:10:28.404046+0000","last_peered":"2024-08-16T19:10:28.404046+0000","last_clean":"2024-08-16T19:10:28.404046+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:10:28.404046+0000","last_undegraded":"2024-08-16T19:10:28.404046+0000","last_fullsized":"2024-08-16T19:10:28.404046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415390+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:10:28.415390+0000","last_peered":"2024-08-16T19:10:28.415390+0000","last_clean":"2024-08-16T19:10:28.415390+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:10:28.415390+0000","last_undegraded":"2024-08-16T19:10:28.415390+0000","last_fullsized":"2024-08-16T19:10:28.415390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528239+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:10:28.528239+0000","last_peered":"2024-08-16T19:10:28.528239+0000","last_clean":"2024-08-16T19:10:28.528239+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:10:28.528239+0000","last_undegraded":"2024-08-16T19:10:28.528239+0000","last_fullsized":"2024-08-16T19:10:28.528239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395532+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:10:28.395532+0000","last_peered":"2024-08-16T19:10:28.395532+0000","last_clean":"2024-08-16T19:10:28.395532+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:10:28.395532+0000","last_undegraded":"2024-08-16T19:10:28.395532+0000","last_fullsized":"2024-08-16T19:10:28.395532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528368+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:10:28.528368+0000","last_peered":"2024-08-16T19:10:28.528368+0000","last_clean":"2024-08-16T19:10:28.528368+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:10:28.528368+0000","last_undegraded":"2024-08-16T19:10:28.528368+0000","last_fullsized":"2024-08-16T19:10:28.528368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415553+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:10:28.415553+0000","last_peered":"2024-08-16T19:10:28.415553+0000","last_clean":"2024-08-16T19:10:28.415553+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:10:28.415553+0000","last_undegraded":"2024-08-16T19:10:28.415553+0000","last_fullsized":"2024-08-16T19:10:28.415553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400776+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:10:28.400776+0000","last_peered":"2024-08-16T19:10:28.400776+0000","last_clean":"2024-08-16T19:10:28.400776+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:10:28.400776+0000","last_undegraded":"2024-08-16T19:10:28.400776+0000","last_fullsized":"2024-08-16T19:10:28.400776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"93'52","reported_seq":116,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.530924+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:10:28.530924+0000","last_peered":"2024-08-16T19:10:28.530924+0000","last_clean":"2024-08-16T19:10:28.530924+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:10:28.530924+0000","last_undegraded":"2024-08-16T19:10:28.530924+0000","last_fullsized":"2024-08-16T19:10:28.530924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":93,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404310+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:10:28.404310+0000","last_peered":"2024-08-16T19:10:28.404310+0000","last_clean":"2024-08-16T19:10:28.404310+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:10:28.404310+0000","last_undegraded":"2024-08-16T19:10:28.404310+0000","last_fullsized":"2024-08-16T19:10:28.404310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309239+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:10:28.309239+0000","last_peered":"2024-08-16T19:10:28.309239+0000","last_clean":"2024-08-16T19:10:28.309239+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:10:28.309239+0000","last_undegraded":"2024-08-16T19:10:28.309239+0000","last_fullsized":"2024-08-16T19:10:28.309239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404515+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:10:28.404515+0000","last_peered":"2024-08-16T19:10:28.404515+0000","last_clean":"2024-08-16T19:10:28.404515+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:10:28.404515+0000","last_undegraded":"2024-08-16T19:10:28.404515+0000","last_fullsized":"2024-08-16T19:10:28.404515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416028+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:10:28.416028+0000","last_peered":"2024-08-16T19:10:28.416028+0000","last_clean":"2024-08-16T19:10:28.416028+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:10:28.416028+0000","last_undegraded":"2024-08-16T19:10:28.416028+0000","last_fullsized":"2024-08-16T19:10:28.416028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308802+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:10:28.308802+0000","last_peered":"2024-08-16T19:10:28.308802+0000","last_clean":"2024-08-16T19:10:28.308802+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:10:28.308802+0000","last_undegraded":"2024-08-16T19:10:28.308802+0000","last_fullsized":"2024-08-16T19:10:28.308802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401633+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:10:28.401633+0000","last_peered":"2024-08-16T19:10:28.401633+0000","last_clean":"2024-08-16T19:10:28.401633+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:10:28.401633+0000","last_undegraded":"2024-08-16T19:10:28.401633+0000","last_fullsized":"2024-08-16T19:10:28.401633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404063+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:10:28.404063+0000","last_peered":"2024-08-16T19:10:28.404063+0000","last_clean":"2024-08-16T19:10:28.404063+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:10:28.404063+0000","last_undegraded":"2024-08-16T19:10:28.404063+0000","last_fullsized":"2024-08-16T19:10:28.404063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308652+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:10:28.308652+0000","last_peered":"2024-08-16T19:10:28.308652+0000","last_clean":"2024-08-16T19:10:28.308652+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:10:28.308652+0000","last_undegraded":"2024-08-16T19:10:28.308652+0000","last_fullsized":"2024-08-16T19:10:28.308652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'8","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.559614+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:10:28.559614+0000","last_peered":"2024-08-16T19:10:28.559614+0000","last_clean":"2024-08-16T19:10:28.559614+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:10:28.559614+0000","last_undegraded":"2024-08-16T19:10:28.559614+0000","last_fullsized":"2024-08-16T19:10:28.559614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309099+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:10:28.309099+0000","last_peered":"2024-08-16T19:10:28.309099+0000","last_clean":"2024-08-16T19:10:28.309099+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:10:28.309099+0000","last_undegraded":"2024-08-16T19:10:28.309099+0000","last_fullsized":"2024-08-16T19:10:28.309099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311486+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:10:28.311486+0000","last_peered":"2024-08-16T19:10:28.311486+0000","last_clean":"2024-08-16T19:10:28.311486+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:10:28.311486+0000","last_undegraded":"2024-08-16T19:10:28.311486+0000","last_fullsized":"2024-08-16T19:10:28.311486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308957+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:10:28.308957+0000","last_peered":"2024-08-16T19:10:28.308957+0000","last_clean":"2024-08-16T19:10:28.308957+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:10:28.308957+0000","last_undegraded":"2024-08-16T19:10:28.308957+0000","last_fullsized":"2024-08-16T19:10:28.308957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309352+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:10:28.309352+0000","last_peered":"2024-08-16T19:10:28.309352+0000","last_clean":"2024-08-16T19:10:28.309352+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:10:28.309352+0000","last_undegraded":"2024-08-16T19:10:28.309352+0000","last_fullsized":"2024-08-16T19:10:28.309352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308734+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:10:28.308734+0000","last_peered":"2024-08-16T19:10:28.308734+0000","last_clean":"2024-08-16T19:10:28.308734+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:10:28.308734+0000","last_undegraded":"2024-08-16T19:10:28.308734+0000","last_fullsized":"2024-08-16T19:10:28.308734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415737+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:10:28.415737+0000","last_peered":"2024-08-16T19:10:28.415737+0000","last_clean":"2024-08-16T19:10:28.415737+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:10:28.415737+0000","last_undegraded":"2024-08-16T19:10:28.415737+0000","last_fullsized":"2024-08-16T19:10:28.415737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312134+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:10:28.312134+0000","last_peered":"2024-08-16T19:10:28.312134+0000","last_clean":"2024-08-16T19:10:28.312134+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:10:28.312134+0000","last_undegraded":"2024-08-16T19:10:28.312134+0000","last_fullsized":"2024-08-16T19:10:28.312134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405223+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:10:28.405223+0000","last_peered":"2024-08-16T19:10:28.405223+0000","last_clean":"2024-08-16T19:10:28.405223+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:10:28.405223+0000","last_undegraded":"2024-08-16T19:10:28.405223+0000","last_fullsized":"2024-08-16T19:10:28.405223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528437+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:10:28.528437+0000","last_peered":"2024-08-16T19:10:28.528437+0000","last_clean":"2024-08-16T19:10:28.528437+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:10:28.528437+0000","last_undegraded":"2024-08-16T19:10:28.528437+0000","last_fullsized":"2024-08-16T19:10:28.528437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415886+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:10:28.415886+0000","last_peered":"2024-08-16T19:10:28.415886+0000","last_clean":"2024-08-16T19:10:28.415886+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:10:28.415886+0000","last_undegraded":"2024-08-16T19:10:28.415886+0000","last_fullsized":"2024-08-16T19:10:28.415886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416317+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:10:28.416317+0000","last_peered":"2024-08-16T19:10:28.416317+0000","last_clean":"2024-08-16T19:10:28.416317+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:10:28.416317+0000","last_undegraded":"2024-08-16T19:10:28.416317+0000","last_fullsized":"2024-08-16T19:10:28.416317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416497+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:10:28.416497+0000","last_peered":"2024-08-16T19:10:28.416497+0000","last_clean":"2024-08-16T19:10:28.416497+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:10:28.416497+0000","last_undegraded":"2024-08-16T19:10:28.416497+0000","last_fullsized":"2024-08-16T19:10:28.416497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'8","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.585430+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:10:28.585430+0000","last_peered":"2024-08-16T19:10:28.585430+0000","last_clean":"2024-08-16T19:10:28.585430+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:10:28.585430+0000","last_undegraded":"2024-08-16T19:10:28.585430+0000","last_fullsized":"2024-08-16T19:10:28.585430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2091,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415428+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:10:28.415428+0000","last_peered":"2024-08-16T19:10:28.415428+0000","last_clean":"2024-08-16T19:10:28.415428+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:10:28.415428+0000","last_undegraded":"2024-08-16T19:10:28.415428+0000","last_fullsized":"2024-08-16T19:10:28.415428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416389+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:10:28.416389+0000","last_peered":"2024-08-16T19:10:28.416389+0000","last_clean":"2024-08-16T19:10:28.416389+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:10:28.416389+0000","last_undegraded":"2024-08-16T19:10:28.416389+0000","last_fullsized":"2024-08-16T19:10:28.416389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'41","reported_seq":90,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.524624+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:10:28.524624+0000","last_peered":"2024-08-16T19:10:28.524624+0000","last_clean":"2024-08-16T19:10:28.524624+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:10:28.524624+0000","last_undegraded":"2024-08-16T19:10:28.524624+0000","last_fullsized":"2024-08-16T19:10:28.524624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395202+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:10:28.395202+0000","last_peered":"2024-08-16T19:10:28.395202+0000","last_clean":"2024-08-16T19:10:28.395202+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:10:28.395202+0000","last_undegraded":"2024-08-16T19:10:28.395202+0000","last_fullsized":"2024-08-16T19:10:28.395202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415903+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:10:28.415903+0000","last_peered":"2024-08-16T19:10:28.415903+0000","last_clean":"2024-08-16T19:10:28.415903+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:10:28.415903+0000","last_undegraded":"2024-08-16T19:10:28.415903+0000","last_fullsized":"2024-08-16T19:10:28.415903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309754+0000","last_change":"2024-08-16T19:09:59.590142+0000","last_active":"2024-08-16T19:10:28.309754+0000","last_peered":"2024-08-16T19:10:28.309754+0000","last_clean":"2024-08-16T19:10:28.309754+0000","last_became_active":"2024-08-16T19:09:59.580675+0000","last_became_peered":"2024-08-16T19:09:59.580675+0000","last_unstale":"2024-08-16T19:10:28.309754+0000","last_undegraded":"2024-08-16T19:10:28.309754+0000","last_fullsized":"2024-08-16T19:10:28.309754+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:06:11.306822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311414+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:10:28.311414+0000","last_peered":"2024-08-16T19:10:28.311414+0000","last_clean":"2024-08-16T19:10:28.311414+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:10:28.311414+0000","last_undegraded":"2024-08-16T19:10:28.311414+0000","last_fullsized":"2024-08-16T19:10:28.311414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416700+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:10:28.416700+0000","last_peered":"2024-08-16T19:10:28.416700+0000","last_clean":"2024-08-16T19:10:28.416700+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:10:28.416700+0000","last_undegraded":"2024-08-16T19:10:28.416700+0000","last_fullsized":"2024-08-16T19:10:28.416700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416400+0000","last_change":"2024-08-16T19:09:59.587138+0000","last_active":"2024-08-16T19:10:28.416400+0000","last_peered":"2024-08-16T19:10:28.416400+0000","last_clean":"2024-08-16T19:10:28.416400+0000","last_became_active":"2024-08-16T19:09:59.579143+0000","last_became_peered":"2024-08-16T19:09:59.579143+0000","last_unstale":"2024-08-16T19:10:28.416400+0000","last_undegraded":"2024-08-16T19:10:28.416400+0000","last_fullsized":"2024-08-16T19:10:28.416400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:08.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311491+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:10:28.311491+0000","last_peered":"2024-08-16T19:10:28.311491+0000","last_clean":"2024-08-16T19:10:28.311491+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:10:28.311491+0000","last_undegraded":"2024-08-16T19:10:28.311491+0000","last_fullsized":"2024-08-16T19:10:28.311491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311658+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:10:28.311658+0000","last_peered":"2024-08-16T19:10:28.311658+0000","last_clean":"2024-08-16T19:10:28.311658+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:10:28.311658+0000","last_undegraded":"2024-08-16T19:10:28.311658+0000","last_fullsized":"2024-08-16T19:10:28.311658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404122+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:10:28.404122+0000","last_peered":"2024-08-16T19:10:28.404122+0000","last_clean":"2024-08-16T19:10:28.404122+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:10:28.404122+0000","last_undegraded":"2024-08-16T19:10:28.404122+0000","last_fullsized":"2024-08-16T19:10:28.404122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309655+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:10:28.309655+0000","last_peered":"2024-08-16T19:10:28.309655+0000","last_clean":"2024-08-16T19:10:28.309655+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:10:28.309655+0000","last_undegraded":"2024-08-16T19:10:28.309655+0000","last_fullsized":"2024-08-16T19:10:28.309655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415689+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:10:28.415689+0000","last_peered":"2024-08-16T19:10:28.415689+0000","last_clean":"2024-08-16T19:10:28.415689+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:10:28.415689+0000","last_undegraded":"2024-08-16T19:10:28.415689+0000","last_fullsized":"2024-08-16T19:10:28.415689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.552691+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:10:28.552691+0000","last_peered":"2024-08-16T19:10:28.552691+0000","last_clean":"2024-08-16T19:10:28.552691+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:10:28.552691+0000","last_undegraded":"2024-08-16T19:10:28.552691+0000","last_fullsized":"2024-08-16T19:10:28.552691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404506+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:10:28.404506+0000","last_peered":"2024-08-16T19:10:28.404506+0000","last_clean":"2024-08-16T19:10:28.404506+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:10:28.404506+0000","last_undegraded":"2024-08-16T19:10:28.404506+0000","last_fullsized":"2024-08-16T19:10:28.404506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415670+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:10:28.415670+0000","last_peered":"2024-08-16T19:10:28.415670+0000","last_clean":"2024-08-16T19:10:28.415670+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:10:28.415670+0000","last_undegraded":"2024-08-16T19:10:28.415670+0000","last_fullsized":"2024-08-16T19:10:28.415670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311811+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:10:28.311811+0000","last_peered":"2024-08-16T19:10:28.311811+0000","last_clean":"2024-08-16T19:10:28.311811+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:10:28.311811+0000","last_undegraded":"2024-08-16T19:10:28.311811+0000","last_fullsized":"2024-08-16T19:10:28.311811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416055+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:10:28.416055+0000","last_peered":"2024-08-16T19:10:28.416055+0000","last_clean":"2024-08-16T19:10:28.416055+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:10:28.416055+0000","last_undegraded":"2024-08-16T19:10:28.416055+0000","last_fullsized":"2024-08-16T19:10:28.416055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308976+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:10:28.308976+0000","last_peered":"2024-08-16T19:10:28.308976+0000","last_clean":"2024-08-16T19:10:28.308976+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:10:28.308976+0000","last_undegraded":"2024-08-16T19:10:28.308976+0000","last_fullsized":"2024-08-16T19:10:28.308976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416259+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:10:28.416259+0000","last_peered":"2024-08-16T19:10:28.416259+0000","last_clean":"2024-08-16T19:10:28.416259+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:10:28.416259+0000","last_undegraded":"2024-08-16T19:10:28.416259+0000","last_fullsized":"2024-08-16T19:10:28.416259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308667+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:10:28.308667+0000","last_peered":"2024-08-16T19:10:28.308667+0000","last_clean":"2024-08-16T19:10:28.308667+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:10:28.308667+0000","last_undegraded":"2024-08-16T19:10:28.308667+0000","last_fullsized":"2024-08-16T19:10:28.308667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404420+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:10:28.404420+0000","last_peered":"2024-08-16T19:10:28.404420+0000","last_clean":"2024-08-16T19:10:28.404420+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:10:28.404420+0000","last_undegraded":"2024-08-16T19:10:28.404420+0000","last_fullsized":"2024-08-16T19:10:28.404420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":298,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.564088+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:10:28.564088+0000","last_peered":"2024-08-16T19:10:28.564088+0000","last_clean":"2024-08-16T19:10:28.564088+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:10:28.564088+0000","last_undegraded":"2024-08-16T19:10:28.564088+0000","last_fullsized":"2024-08-16T19:10:28.564088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.399989+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:10:28.399989+0000","last_peered":"2024-08-16T19:10:28.399989+0000","last_clean":"2024-08-16T19:10:28.399989+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:10:28.399989+0000","last_undegraded":"2024-08-16T19:10:28.399989+0000","last_fullsized":"2024-08-16T19:10:28.399989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415798+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:10:28.415798+0000","last_peered":"2024-08-16T19:10:28.415798+0000","last_clean":"2024-08-16T19:10:28.415798+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:10:28.415798+0000","last_undegraded":"2024-08-16T19:10:28.415798+0000","last_fullsized":"2024-08-16T19:10:28.415798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312030+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:10:28.312030+0000","last_peered":"2024-08-16T19:10:28.312030+0000","last_clean":"2024-08-16T19:10:28.312030+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:10:28.312030+0000","last_undegraded":"2024-08-16T19:10:28.312030+0000","last_fullsized":"2024-08-16T19:10:28.312030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528335+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:10:28.528335+0000","last_peered":"2024-08-16T19:10:28.528335+0000","last_clean":"2024-08-16T19:10:28.528335+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:10:28.528335+0000","last_undegraded":"2024-08-16T19:10:28.528335+0000","last_fullsized":"2024-08-16T19:10:28.528335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395911+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:10:28.395911+0000","last_peered":"2024-08-16T19:10:28.395911+0000","last_clean":"2024-08-16T19:10:28.395911+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:10:28.395911+0000","last_undegraded":"2024-08-16T19:10:28.395911+0000","last_fullsized":"2024-08-16T19:10:28.395911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416395+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:10:28.416395+0000","last_peered":"2024-08-16T19:10:28.416395+0000","last_clean":"2024-08-16T19:10:28.416395+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:10:28.416395+0000","last_undegraded":"2024-08-16T19:10:28.416395+0000","last_fullsized":"2024-08-16T19:10:28.416395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.399854+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:10:28.399854+0000","last_peered":"2024-08-16T19:10:28.399854+0000","last_clean":"2024-08-16T19:10:28.399854+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:10:28.399854+0000","last_undegraded":"2024-08-16T19:10:28.399854+0000","last_fullsized":"2024-08-16T19:10:28.399854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416283+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:10:28.416283+0000","last_peered":"2024-08-16T19:10:28.416283+0000","last_clean":"2024-08-16T19:10:28.416283+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:10:28.416283+0000","last_undegraded":"2024-08-16T19:10:28.416283+0000","last_fullsized":"2024-08-16T19:10:28.416283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":252,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.584792+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:10:28.584792+0000","last_peered":"2024-08-16T19:10:28.584792+0000","last_clean":"2024-08-16T19:10:28.584792+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:10:28.584792+0000","last_undegraded":"2024-08-16T19:10:28.584792+0000","last_fullsized":"2024-08-16T19:10:28.584792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404850+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:10:28.404850+0000","last_peered":"2024-08-16T19:10:28.404850+0000","last_clean":"2024-08-16T19:10:28.404850+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:10:28.404850+0000","last_undegraded":"2024-08-16T19:10:28.404850+0000","last_fullsized":"2024-08-16T19:10:28.404850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395310+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:10:28.395310+0000","last_peered":"2024-08-16T19:10:28.395310+0000","last_clean":"2024-08-16T19:10:28.395310+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:10:28.395310+0000","last_undegraded":"2024-08-16T19:10:28.395310+0000","last_fullsized":"2024-08-16T19:10:28.395310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400012+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:10:28.400012+0000","last_peered":"2024-08-16T19:10:28.400012+0000","last_clean":"2024-08-16T19:10:28.400012+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:10:28.400012+0000","last_undegraded":"2024-08-16T19:10:28.400012+0000","last_fullsized":"2024-08-16T19:10:28.400012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311593+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:10:28.311593+0000","last_peered":"2024-08-16T19:10:28.311593+0000","last_clean":"2024-08-16T19:10:28.311593+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:10:28.311593+0000","last_undegraded":"2024-08-16T19:10:28.311593+0000","last_fullsized":"2024-08-16T19:10:28.311593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415754+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:10:28.415754+0000","last_peered":"2024-08-16T19:10:28.415754+0000","last_clean":"2024-08-16T19:10:28.415754+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:10:28.415754+0000","last_undegraded":"2024-08-16T19:10:28.415754+0000","last_fullsized":"2024-08-16T19:10:28.415754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404608+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:10:28.404608+0000","last_peered":"2024-08-16T19:10:28.404608+0000","last_clean":"2024-08-16T19:10:28.404608+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:10:28.404608+0000","last_undegraded":"2024-08-16T19:10:28.404608+0000","last_fullsized":"2024-08-16T19:10:28.404608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308849+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:10:28.308849+0000","last_peered":"2024-08-16T19:10:28.308849+0000","last_clean":"2024-08-16T19:10:28.308849+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:10:28.308849+0000","last_undegraded":"2024-08-16T19:10:28.308849+0000","last_fullsized":"2024-08-16T19:10:28.308849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308584+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:10:28.308584+0000","last_peered":"2024-08-16T19:10:28.308584+0000","last_clean":"2024-08-16T19:10:28.308584+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:10:28.308584+0000","last_undegraded":"2024-08-16T19:10:28.308584+0000","last_fullsized":"2024-08-16T19:10:28.308584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311961+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:10:28.311961+0000","last_peered":"2024-08-16T19:10:28.311961+0000","last_clean":"2024-08-16T19:10:28.311961+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:10:28.311961+0000","last_undegraded":"2024-08-16T19:10:28.311961+0000","last_fullsized":"2024-08-16T19:10:28.311961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416295+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:10:28.416295+0000","last_peered":"2024-08-16T19:10:28.416295+0000","last_clean":"2024-08-16T19:10:28.416295+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:10:28.416295+0000","last_undegraded":"2024-08-16T19:10:28.416295+0000","last_fullsized":"2024-08-16T19:10:28.416295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.567085+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:10:28.567085+0000","last_peered":"2024-08-16T19:10:28.567085+0000","last_clean":"2024-08-16T19:10:28.567085+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:10:28.567085+0000","last_undegraded":"2024-08-16T19:10:28.567085+0000","last_fullsized":"2024-08-16T19:10:28.567085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404156+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:10:28.404156+0000","last_peered":"2024-08-16T19:10:28.404156+0000","last_clean":"2024-08-16T19:10:28.404156+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:10:28.404156+0000","last_undegraded":"2024-08-16T19:10:28.404156+0000","last_fullsized":"2024-08-16T19:10:28.404156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416359+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:10:28.416359+0000","last_peered":"2024-08-16T19:10:28.416359+0000","last_clean":"2024-08-16T19:10:28.416359+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:10:28.416359+0000","last_undegraded":"2024-08-16T19:10:28.416359+0000","last_fullsized":"2024-08-16T19:10:28.416359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415784+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:10:28.415784+0000","last_peered":"2024-08-16T19:10:28.415784+0000","last_clean":"2024-08-16T19:10:28.415784+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:10:28.415784+0000","last_undegraded":"2024-08-16T19:10:28.415784+0000","last_fullsized":"2024-08-16T19:10:28.415784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415826+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:10:28.415826+0000","last_peered":"2024-08-16T19:10:28.415826+0000","last_clean":"2024-08-16T19:10:28.415826+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:10:28.415826+0000","last_undegraded":"2024-08-16T19:10:28.415826+0000","last_fullsized":"2024-08-16T19:10:28.415826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415629+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:10:28.415629+0000","last_peered":"2024-08-16T19:10:28.415629+0000","last_clean":"2024-08-16T19:10:28.415629+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:10:28.415629+0000","last_undegraded":"2024-08-16T19:10:28.415629+0000","last_fullsized":"2024-08-16T19:10:28.415629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395262+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:10:28.395262+0000","last_peered":"2024-08-16T19:10:28.395262+0000","last_clean":"2024-08-16T19:10:28.395262+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:10:28.395262+0000","last_undegraded":"2024-08-16T19:10:28.395262+0000","last_fullsized":"2024-08-16T19:10:28.395262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309213+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:10:28.309213+0000","last_peered":"2024-08-16T19:10:28.309213+0000","last_clean":"2024-08-16T19:10:28.309213+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:10:28.309213+0000","last_undegraded":"2024-08-16T19:10:28.309213+0000","last_fullsized":"2024-08-16T19:10:28.309213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416309+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:10:28.416309+0000","last_peered":"2024-08-16T19:10:28.416309+0000","last_clean":"2024-08-16T19:10:28.416309+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:10:28.416309+0000","last_undegraded":"2024-08-16T19:10:28.416309+0000","last_fullsized":"2024-08-16T19:10:28.416309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416854+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:10:28.416854+0000","last_peered":"2024-08-16T19:10:28.416854+0000","last_clean":"2024-08-16T19:10:28.416854+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:10:28.416854+0000","last_undegraded":"2024-08-16T19:10:28.416854+0000","last_fullsized":"2024-08-16T19:10:28.416854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406058+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:10:28.406058+0000","last_peered":"2024-08-16T19:10:28.406058+0000","last_clean":"2024-08-16T19:10:28.406058+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:10:28.406058+0000","last_undegraded":"2024-08-16T19:10:28.406058+0000","last_fullsized":"2024-08-16T19:10:28.406058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415471+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:10:28.415471+0000","last_peered":"2024-08-16T19:10:28.415471+0000","last_clean":"2024-08-16T19:10:28.415471+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:10:28.415471+0000","last_undegraded":"2024-08-16T19:10:28.415471+0000","last_fullsized":"2024-08-16T19:10:28.415471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405067+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:10:28.405067+0000","last_peered":"2024-08-16T19:10:28.405067+0000","last_clean":"2024-08-16T19:10:28.405067+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:10:28.405067+0000","last_undegraded":"2024-08-16T19:10:28.405067+0000","last_fullsized":"2024-08-16T19:10:28.405067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415591+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:10:28.415591+0000","last_peered":"2024-08-16T19:10:28.415591+0000","last_clean":"2024-08-16T19:10:28.415591+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:10:28.415591+0000","last_undegraded":"2024-08-16T19:10:28.415591+0000","last_fullsized":"2024-08-16T19:10:28.415591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309143+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:10:28.309143+0000","last_peered":"2024-08-16T19:10:28.309143+0000","last_clean":"2024-08-16T19:10:28.309143+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:10:28.309143+0000","last_undegraded":"2024-08-16T19:10:28.309143+0000","last_fullsized":"2024-08-16T19:10:28.309143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416085+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:10:28.416085+0000","last_peered":"2024-08-16T19:10:28.416085+0000","last_clean":"2024-08-16T19:10:28.416085+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:10:28.416085+0000","last_undegraded":"2024-08-16T19:10:28.416085+0000","last_fullsized":"2024-08-16T19:10:28.416085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416507+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:10:28.416507+0000","last_peered":"2024-08-16T19:10:28.416507+0000","last_clean":"2024-08-16T19:10:28.416507+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:10:28.416507+0000","last_undegraded":"2024-08-16T19:10:28.416507+0000","last_fullsized":"2024-08-16T19:10:28.416507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401991+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:10:28.401991+0000","last_peered":"2024-08-16T19:10:28.401991+0000","last_clean":"2024-08-16T19:10:28.401991+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:10:28.401991+0000","last_undegraded":"2024-08-16T19:10:28.401991+0000","last_fullsized":"2024-08-16T19:10:28.401991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416849+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:10:28.416849+0000","last_peered":"2024-08-16T19:10:28.416849+0000","last_clean":"2024-08-16T19:10:28.416849+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:10:28.416849+0000","last_undegraded":"2024-08-16T19:10:28.416849+0000","last_fullsized":"2024-08-16T19:10:28.416849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415320+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:10:28.415320+0000","last_peered":"2024-08-16T19:10:28.415320+0000","last_clean":"2024-08-16T19:10:28.415320+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:10:28.415320+0000","last_undegraded":"2024-08-16T19:10:28.415320+0000","last_fullsized":"2024-08-16T19:10:28.415320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.394824+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:10:28.394824+0000","last_peered":"2024-08-16T19:10:28.394824+0000","last_clean":"2024-08-16T19:10:28.394824+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:10:28.394824+0000","last_undegraded":"2024-08-16T19:10:28.394824+0000","last_fullsized":"2024-08-16T19:10:28.394824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395082+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:10:28.395082+0000","last_peered":"2024-08-16T19:10:28.395082+0000","last_clean":"2024-08-16T19:10:28.395082+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:10:28.395082+0000","last_undegraded":"2024-08-16T19:10:28.395082+0000","last_fullsized":"2024-08-16T19:10:28.395082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309135+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:10:28.309135+0000","last_peered":"2024-08-16T19:10:28.309135+0000","last_clean":"2024-08-16T19:10:28.309135+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:10:28.309135+0000","last_undegraded":"2024-08-16T19:10:28.309135+0000","last_fullsized":"2024-08-16T19:10:28.309135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528358+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:10:28.528358+0000","last_peered":"2024-08-16T19:10:28.528358+0000","last_clean":"2024-08-16T19:10:28.528358+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:10:28.528358+0000","last_undegraded":"2024-08-16T19:10:28.528358+0000","last_fullsized":"2024-08-16T19:10:28.528358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416564+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:10:28.416564+0000","last_peered":"2024-08-16T19:10:28.416564+0000","last_clean":"2024-08-16T19:10:28.416564+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:10:28.416564+0000","last_undegraded":"2024-08-16T19:10:28.416564+0000","last_fullsized":"2024-08-16T19:10:28.416564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309002+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:10:28.309002+0000","last_peered":"2024-08-16T19:10:28.309002+0000","last_clean":"2024-08-16T19:10:28.309002+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:10:28.309002+0000","last_undegraded":"2024-08-16T19:10:28.309002+0000","last_fullsized":"2024-08-16T19:10:28.309002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416294+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:10:28.416294+0000","last_peered":"2024-08-16T19:10:28.416294+0000","last_clean":"2024-08-16T19:10:28.416294+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:10:28.416294+0000","last_undegraded":"2024-08-16T19:10:28.416294+0000","last_fullsized":"2024-08-16T19:10:28.416294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404467+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:10:28.404467+0000","last_peered":"2024-08-16T19:10:28.404467+0000","last_clean":"2024-08-16T19:10:28.404467+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:10:28.404467+0000","last_undegraded":"2024-08-16T19:10:28.404467+0000","last_fullsized":"2024-08-16T19:10:28.404467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416036+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:10:28.416036+0000","last_peered":"2024-08-16T19:10:28.416036+0000","last_clean":"2024-08-16T19:10:28.416036+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:10:28.416036+0000","last_undegraded":"2024-08-16T19:10:28.416036+0000","last_fullsized":"2024-08-16T19:10:28.416036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308504+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:10:28.308504+0000","last_peered":"2024-08-16T19:10:28.308504+0000","last_clean":"2024-08-16T19:10:28.308504+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:10:28.308504+0000","last_undegraded":"2024-08-16T19:10:28.308504+0000","last_fullsized":"2024-08-16T19:10:28.308504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404114+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:10:28.404114+0000","last_peered":"2024-08-16T19:10:28.404114+0000","last_clean":"2024-08-16T19:10:28.404114+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:10:28.404114+0000","last_undegraded":"2024-08-16T19:10:28.404114+0000","last_fullsized":"2024-08-16T19:10:28.404114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528254+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:10:28.528254+0000","last_peered":"2024-08-16T19:10:28.528254+0000","last_clean":"2024-08-16T19:10:28.528254+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:10:28.528254+0000","last_undegraded":"2024-08-16T19:10:28.528254+0000","last_fullsized":"2024-08-16T19:10:28.528254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395415+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:10:28.395415+0000","last_peered":"2024-08-16T19:10:28.395415+0000","last_clean":"2024-08-16T19:10:28.395415+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:10:28.395415+0000","last_undegraded":"2024-08-16T19:10:28.395415+0000","last_fullsized":"2024-08-16T19:10:28.395415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395414+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:10:28.395414+0000","last_peered":"2024-08-16T19:10:28.395414+0000","last_clean":"2024-08-16T19:10:28.395414+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:10:28.395414+0000","last_undegraded":"2024-08-16T19:10:28.395414+0000","last_fullsized":"2024-08-16T19:10:28.395414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416642+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:10:28.416642+0000","last_peered":"2024-08-16T19:10:28.416642+0000","last_clean":"2024-08-16T19:10:28.416642+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:10:28.416642+0000","last_undegraded":"2024-08-16T19:10:28.416642+0000","last_fullsized":"2024-08-16T19:10:28.416642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603171,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1160,"num_read_kb":694,"num_write":385,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1809513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":538317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28948,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716204,"statfs":{"total":95995035648,"available":95965392896,"internally_reserved":0,"allocated":1916928,"data_stored":1325101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939699,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":541400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1490944,"data_stored":930876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1136020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":533729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1843200,"data_stored":1260970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":537413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1057332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":597216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":603539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":535185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:10:32.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:10:32.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: [16/Aug/2024:19:10:31] ENGINE Serving on http://172.21.15.62:8765 2024-08-16T19:10:32.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: [16/Aug/2024:19:10:31] ENGINE Serving on https://172.21.15.62:7150 2024-08-16T19:10:32.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: [16/Aug/2024:19:10:31] ENGINE Bus STARTED 2024-08-16T19:10:32.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: [16/Aug/2024:19:10:31] ENGINE Error in HTTPServer.serve 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: Traceback (most recent call last): 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._connections.run(self.expiration_interval) 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._run(expiration_interval) 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: s = self.context.wrap_socket( 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: return self.sslsocket_class._create( 2024-08-16T19:10:32.580 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout: self.do_handshake() 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout: self._sslobj.do_handshake() 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:32.581 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:32 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: [16/Aug/2024:19:10:31] ENGINE Serving on http://172.21.15.62:8765 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: [16/Aug/2024:19:10:31] ENGINE Serving on https://172.21.15.62:7150 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: [16/Aug/2024:19:10:31] ENGINE Bus STARTED 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: [16/Aug/2024:19:10:31] ENGINE Error in HTTPServer.serve 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout: Traceback (most recent call last): 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:10:32.638 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._connections.run(self.expiration_interval) 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._run(expiration_interval) 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: s = self.context.wrap_socket( 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: return self.sslsocket_class._create( 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: self.do_handshake() 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:10:32.639 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._sslobj.do_handshake() 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:32 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:32.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: [16/Aug/2024:19:10:31] ENGINE Serving on http://172.21.15.62:8765 2024-08-16T19:10:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: [16/Aug/2024:19:10:31] ENGINE Serving on https://172.21.15.62:7150 2024-08-16T19:10:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: [16/Aug/2024:19:10:31] ENGINE Bus STARTED 2024-08-16T19:10:32.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: [16/Aug/2024:19:10:31] ENGINE Error in HTTPServer.serve 2024-08-16T19:10:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout: Traceback (most recent call last): 2024-08-16T19:10:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-16T19:10:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._connections.run(self.expiration_interval) 2024-08-16T19:10:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-16T19:10:32.749 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._run(expiration_interval) 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: s = self.context.wrap_socket( 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: return self.sslsocket_class._create( 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: self.do_handshake() 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: self._sslobj.do_handshake() 2024-08-16T19:10:32.750 INFO:journalctl@ceph.mon.a.smithi062.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi062:/etc/ceph/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi155:/etc/ceph/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi161:/etc/ceph/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-08-16T19:10:32.751 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: Updating smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:32.752 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:32 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.155+0000 7f8ab2760640 1 -- 172.21.15.62:0/3494270149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac1034b0 msgr2=0x7f8aac1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.155+0000 7f8ab2760640 1 --2- 172.21.15.62:0/3494270149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac1034b0 0x7f8aac1078f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009930 tx=0x7f8aa002a420 comp rx=0 tx=0).stop 2024-08-16T19:10:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.155+0000 7f8ab2760640 1 -- 172.21.15.62:0/3494270149 shutdown_connections 2024-08-16T19:10:33.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.155+0000 7f8ab2760640 1 --2- 172.21.15.62:0/3494270149 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac108030 0x7f8aac10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.155+0000 7f8ab2760640 1 --2- 172.21.15.62:0/3494270149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac1034b0 0x7f8aac1078f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 --2- 172.21.15.62:0/3494270149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac102b00 0x7f8aac102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 -- 172.21.15.62:0/3494270149 >> 172.21.15.62:0/3494270149 conn(0x7f8aac0fc500 msgr2=0x7f8aac0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 -- 172.21.15.62:0/3494270149 shutdown_connections 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 -- 172.21.15.62:0/3494270149 wait complete. 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 Processor -- start 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.156+0000 7f8ab2760640 1 -- start start 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac102b00 0x7f8aac1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 0x7f8aac1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac111e00 con 0x7f8aac108030 2024-08-16T19:10:33.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac111c80 con 0x7f8aac1034b0 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac111f80 con 0x7f8aac102b00 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44300/0 (socket says 172.21.15.62:44300) 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 -- 172.21.15.62:0/462951391 learned_addr learned my addr 172.21.15.62:0/462951391 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aabfff640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac102b00 0x7f8aac1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aab7fe640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 0x7f8aac1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 -- 172.21.15.62:0/462951391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac102b00 msgr2=0x7f8aac1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac102b00 0x7f8aac1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 -- 172.21.15.62:0/462951391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 msgr2=0x7f8aac1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 0x7f8aac1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 -- 172.21.15.62:0/462951391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac19d250 con 0x7f8aac108030 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aab7fe640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 0x7f8aac1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab0cfd640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cce0 tx=0x7f8aa400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:33.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400f450 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa400fad0 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac19d540 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.157+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4018c10 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.158+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac1a4e20 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.158+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac102ee0 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.160+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f8aa400f5f0 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.161+0000 7f8aa97fa640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 0x7f8a940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.161+0000 7f8aabfff640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 0x7f8a940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.161+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f8aa40cfe10 con 0x7f8aac108030 2024-08-16T19:10:33.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.161+0000 7f8aabfff640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 0x7f8a940a51b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0058c0 tx=0x7f8a9c00a2d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:33.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.165+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8aa4014030 con 0x7f8aac108030 2024-08-16T19:10:33.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.295+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8aa40cd450 con 0x7f8aac108030 2024-08-16T19:10:33.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.343+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8aac062360 con 0x7f8a940a2cf0 2024-08-16T19:10:33.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.352+0000 7f8aa97fa640 1 -- 172.21.15.62:0/462951391 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339436 (secure 0 0 0) 0x7f8aac062360 con 0x7f8a940a2cf0 2024-08-16T19:10:33.356 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:33.357 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:10:33.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 msgr2=0x7f8a940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:33.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 0x7f8a940a51b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0058c0 tx=0x7f8a9c00a2d0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 msgr2=0x7f8aac19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:33.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cce0 tx=0x7f8aa400a620 comp rx=0 tx=0).stop 2024-08-16T19:10:33.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 shutdown_connections 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8a940a2cf0 0x7f8a940a51b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac108030 0x7f8aac19cb70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8aac1034b0 0x7f8aac1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 --2- 172.21.15.62:0/462951391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aac102b00 0x7f8aac1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 >> 172.21.15.62:0/462951391 conn(0x7f8aac0fc500 msgr2=0x7f8aac105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 shutdown_connections 2024-08-16T19:10:33.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:33.355+0000 7f8ab2760640 1 -- 172.21.15.62:0/462951391 wait complete. 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:33.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:33 smithi155 ceph-mon[37413]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:33.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:33 smithi161 ceph-mon[37576]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-08-16T19:10:33.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:10:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:10:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:10:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:10:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:10:33.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:33 smithi062 ceph-mon[32993]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-08-16T19:10:34.154 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-08-16T19:10:32.321950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193773,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1296,"num_read_kb":813,"num_write":568,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381876,"kb_used_data":15988,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559948,"statfs":{"total":1151940427776,"available":1151549386752,"internally_reserved":0,"allocated":16371712,"data_stored":9597098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990343"},"pg_stats":[{"pgid":"2.1f","version":"93'15","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.558869+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:10:28.558869+0000","last_peered":"2024-08-16T19:10:28.558869+0000","last_clean":"2024-08-16T19:10:28.558869+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:10:28.558869+0000","last_undegraded":"2024-08-16T19:10:28.558869+0000","last_fullsized":"2024-08-16T19:10:28.558869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406289+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:10:28.406289+0000","last_peered":"2024-08-16T19:10:28.406289+0000","last_clean":"2024-08-16T19:10:28.406289+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:10:28.406289+0000","last_undegraded":"2024-08-16T19:10:28.406289+0000","last_fullsized":"2024-08-16T19:10:28.406289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415552+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:10:28.415552+0000","last_peered":"2024-08-16T19:10:28.415552+0000","last_clean":"2024-08-16T19:10:28.415552+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:10:28.415552+0000","last_undegraded":"2024-08-16T19:10:28.415552+0000","last_fullsized":"2024-08-16T19:10:28.415552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406640+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:10:28.406640+0000","last_peered":"2024-08-16T19:10:28.406640+0000","last_clean":"2024-08-16T19:10:28.406640+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:10:28.406640+0000","last_undegraded":"2024-08-16T19:10:28.406640+0000","last_fullsized":"2024-08-16T19:10:28.406640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395855+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:10:28.395855+0000","last_peered":"2024-08-16T19:10:28.395855+0000","last_clean":"2024-08-16T19:10:28.395855+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:10:28.395855+0000","last_undegraded":"2024-08-16T19:10:28.395855+0000","last_fullsized":"2024-08-16T19:10:28.395855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527883+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:10:28.527883+0000","last_peered":"2024-08-16T19:10:28.527883+0000","last_clean":"2024-08-16T19:10:28.527883+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:10:28.527883+0000","last_undegraded":"2024-08-16T19:10:28.527883+0000","last_fullsized":"2024-08-16T19:10:28.527883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404475+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:10:28.404475+0000","last_peered":"2024-08-16T19:10:28.404475+0000","last_clean":"2024-08-16T19:10:28.404475+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:10:28.404475+0000","last_undegraded":"2024-08-16T19:10:28.404475+0000","last_fullsized":"2024-08-16T19:10:28.404475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415713+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:10:28.415713+0000","last_peered":"2024-08-16T19:10:28.415713+0000","last_clean":"2024-08-16T19:10:28.415713+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:10:28.415713+0000","last_undegraded":"2024-08-16T19:10:28.415713+0000","last_fullsized":"2024-08-16T19:10:28.415713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528080+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:10:28.528080+0000","last_peered":"2024-08-16T19:10:28.528080+0000","last_clean":"2024-08-16T19:10:28.528080+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:10:28.528080+0000","last_undegraded":"2024-08-16T19:10:28.528080+0000","last_fullsized":"2024-08-16T19:10:28.528080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404392+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:10:28.404392+0000","last_peered":"2024-08-16T19:10:28.404392+0000","last_clean":"2024-08-16T19:10:28.404392+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:10:28.404392+0000","last_undegraded":"2024-08-16T19:10:28.404392+0000","last_fullsized":"2024-08-16T19:10:28.404392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400420+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:10:28.400420+0000","last_peered":"2024-08-16T19:10:28.400420+0000","last_clean":"2024-08-16T19:10:28.400420+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:10:28.400420+0000","last_undegraded":"2024-08-16T19:10:28.400420+0000","last_fullsized":"2024-08-16T19:10:28.400420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527974+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:10:28.527974+0000","last_peered":"2024-08-16T19:10:28.527974+0000","last_clean":"2024-08-16T19:10:28.527974+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:10:28.527974+0000","last_undegraded":"2024-08-16T19:10:28.527974+0000","last_fullsized":"2024-08-16T19:10:28.527974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416184+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:10:28.416184+0000","last_peered":"2024-08-16T19:10:28.416184+0000","last_clean":"2024-08-16T19:10:28.416184+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:10:28.416184+0000","last_undegraded":"2024-08-16T19:10:28.416184+0000","last_fullsized":"2024-08-16T19:10:28.416184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404284+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:10:28.404284+0000","last_peered":"2024-08-16T19:10:28.404284+0000","last_clean":"2024-08-16T19:10:28.404284+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:10:28.404284+0000","last_undegraded":"2024-08-16T19:10:28.404284+0000","last_fullsized":"2024-08-16T19:10:28.404284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528162+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:10:28.528162+0000","last_peered":"2024-08-16T19:10:28.528162+0000","last_clean":"2024-08-16T19:10:28.528162+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:10:28.528162+0000","last_undegraded":"2024-08-16T19:10:28.528162+0000","last_fullsized":"2024-08-16T19:10:28.528162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404992+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:10:28.404992+0000","last_peered":"2024-08-16T19:10:28.404992+0000","last_clean":"2024-08-16T19:10:28.404992+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:10:28.404992+0000","last_undegraded":"2024-08-16T19:10:28.404992+0000","last_fullsized":"2024-08-16T19:10:28.404992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416087+0000","last_change":"2024-08-16T19:09:59.583331+0000","last_active":"2024-08-16T19:10:28.416087+0000","last_peered":"2024-08-16T19:10:28.416087+0000","last_clean":"2024-08-16T19:10:28.416087+0000","last_became_active":"2024-08-16T19:09:59.578673+0000","last_became_peered":"2024-08-16T19:09:59.578673+0000","last_unstale":"2024-08-16T19:10:28.416087+0000","last_undegraded":"2024-08-16T19:10:28.416087+0000","last_fullsized":"2024-08-16T19:10:28.416087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:00:56.841049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528119+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:10:28.528119+0000","last_peered":"2024-08-16T19:10:28.528119+0000","last_clean":"2024-08-16T19:10:28.528119+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:10:28.528119+0000","last_undegraded":"2024-08-16T19:10:28.528119+0000","last_fullsized":"2024-08-16T19:10:28.528119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415835+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:10:28.415835+0000","last_peered":"2024-08-16T19:10:28.415835+0000","last_clean":"2024-08-16T19:10:28.415835+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:10:28.415835+0000","last_undegraded":"2024-08-16T19:10:28.415835+0000","last_fullsized":"2024-08-16T19:10:28.415835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405227+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:10:28.405227+0000","last_peered":"2024-08-16T19:10:28.405227+0000","last_clean":"2024-08-16T19:10:28.405227+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:10:28.405227+0000","last_undegraded":"2024-08-16T19:10:28.405227+0000","last_fullsized":"2024-08-16T19:10:28.405227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400990+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:10:28.400990+0000","last_peered":"2024-08-16T19:10:28.400990+0000","last_clean":"2024-08-16T19:10:28.400990+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:10:28.400990+0000","last_undegraded":"2024-08-16T19:10:28.400990+0000","last_fullsized":"2024-08-16T19:10:28.400990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404046+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:10:28.404046+0000","last_peered":"2024-08-16T19:10:28.404046+0000","last_clean":"2024-08-16T19:10:28.404046+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:10:28.404046+0000","last_undegraded":"2024-08-16T19:10:28.404046+0000","last_fullsized":"2024-08-16T19:10:28.404046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415390+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:10:28.415390+0000","last_peered":"2024-08-16T19:10:28.415390+0000","last_clean":"2024-08-16T19:10:28.415390+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:10:28.415390+0000","last_undegraded":"2024-08-16T19:10:28.415390+0000","last_fullsized":"2024-08-16T19:10:28.415390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528239+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:10:28.528239+0000","last_peered":"2024-08-16T19:10:28.528239+0000","last_clean":"2024-08-16T19:10:28.528239+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:10:28.528239+0000","last_undegraded":"2024-08-16T19:10:28.528239+0000","last_fullsized":"2024-08-16T19:10:28.528239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395532+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:10:28.395532+0000","last_peered":"2024-08-16T19:10:28.395532+0000","last_clean":"2024-08-16T19:10:28.395532+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:10:28.395532+0000","last_undegraded":"2024-08-16T19:10:28.395532+0000","last_fullsized":"2024-08-16T19:10:28.395532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528368+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:10:28.528368+0000","last_peered":"2024-08-16T19:10:28.528368+0000","last_clean":"2024-08-16T19:10:28.528368+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:10:28.528368+0000","last_undegraded":"2024-08-16T19:10:28.528368+0000","last_fullsized":"2024-08-16T19:10:28.528368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415553+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:10:28.415553+0000","last_peered":"2024-08-16T19:10:28.415553+0000","last_clean":"2024-08-16T19:10:28.415553+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:10:28.415553+0000","last_undegraded":"2024-08-16T19:10:28.415553+0000","last_fullsized":"2024-08-16T19:10:28.415553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400776+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:10:28.400776+0000","last_peered":"2024-08-16T19:10:28.400776+0000","last_clean":"2024-08-16T19:10:28.400776+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:10:28.400776+0000","last_undegraded":"2024-08-16T19:10:28.400776+0000","last_fullsized":"2024-08-16T19:10:28.400776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"93'52","reported_seq":116,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.530924+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:10:28.530924+0000","last_peered":"2024-08-16T19:10:28.530924+0000","last_clean":"2024-08-16T19:10:28.530924+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:10:28.530924+0000","last_undegraded":"2024-08-16T19:10:28.530924+0000","last_fullsized":"2024-08-16T19:10:28.530924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":93,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404310+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:10:28.404310+0000","last_peered":"2024-08-16T19:10:28.404310+0000","last_clean":"2024-08-16T19:10:28.404310+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:10:28.404310+0000","last_undegraded":"2024-08-16T19:10:28.404310+0000","last_fullsized":"2024-08-16T19:10:28.404310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309239+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:10:28.309239+0000","last_peered":"2024-08-16T19:10:28.309239+0000","last_clean":"2024-08-16T19:10:28.309239+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:10:28.309239+0000","last_undegraded":"2024-08-16T19:10:28.309239+0000","last_fullsized":"2024-08-16T19:10:28.309239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404515+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:10:28.404515+0000","last_peered":"2024-08-16T19:10:28.404515+0000","last_clean":"2024-08-16T19:10:28.404515+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:10:28.404515+0000","last_undegraded":"2024-08-16T19:10:28.404515+0000","last_fullsized":"2024-08-16T19:10:28.404515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416028+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:10:28.416028+0000","last_peered":"2024-08-16T19:10:28.416028+0000","last_clean":"2024-08-16T19:10:28.416028+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:10:28.416028+0000","last_undegraded":"2024-08-16T19:10:28.416028+0000","last_fullsized":"2024-08-16T19:10:28.416028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308802+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:10:28.308802+0000","last_peered":"2024-08-16T19:10:28.308802+0000","last_clean":"2024-08-16T19:10:28.308802+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:10:28.308802+0000","last_undegraded":"2024-08-16T19:10:28.308802+0000","last_fullsized":"2024-08-16T19:10:28.308802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401633+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:10:28.401633+0000","last_peered":"2024-08-16T19:10:28.401633+0000","last_clean":"2024-08-16T19:10:28.401633+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:10:28.401633+0000","last_undegraded":"2024-08-16T19:10:28.401633+0000","last_fullsized":"2024-08-16T19:10:28.401633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404063+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:10:28.404063+0000","last_peered":"2024-08-16T19:10:28.404063+0000","last_clean":"2024-08-16T19:10:28.404063+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:10:28.404063+0000","last_undegraded":"2024-08-16T19:10:28.404063+0000","last_fullsized":"2024-08-16T19:10:28.404063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308652+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:10:28.308652+0000","last_peered":"2024-08-16T19:10:28.308652+0000","last_clean":"2024-08-16T19:10:28.308652+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:10:28.308652+0000","last_undegraded":"2024-08-16T19:10:28.308652+0000","last_fullsized":"2024-08-16T19:10:28.308652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'8","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.559614+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:10:28.559614+0000","last_peered":"2024-08-16T19:10:28.559614+0000","last_clean":"2024-08-16T19:10:28.559614+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:10:28.559614+0000","last_undegraded":"2024-08-16T19:10:28.559614+0000","last_fullsized":"2024-08-16T19:10:28.559614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309099+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:10:28.309099+0000","last_peered":"2024-08-16T19:10:28.309099+0000","last_clean":"2024-08-16T19:10:28.309099+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:10:28.309099+0000","last_undegraded":"2024-08-16T19:10:28.309099+0000","last_fullsized":"2024-08-16T19:10:28.309099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311486+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:10:28.311486+0000","last_peered":"2024-08-16T19:10:28.311486+0000","last_clean":"2024-08-16T19:10:28.311486+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:10:28.311486+0000","last_undegraded":"2024-08-16T19:10:28.311486+0000","last_fullsized":"2024-08-16T19:10:28.311486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308957+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:10:28.308957+0000","last_peered":"2024-08-16T19:10:28.308957+0000","last_clean":"2024-08-16T19:10:28.308957+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:10:28.308957+0000","last_undegraded":"2024-08-16T19:10:28.308957+0000","last_fullsized":"2024-08-16T19:10:28.308957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309352+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:10:28.309352+0000","last_peered":"2024-08-16T19:10:28.309352+0000","last_clean":"2024-08-16T19:10:28.309352+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:10:28.309352+0000","last_undegraded":"2024-08-16T19:10:28.309352+0000","last_fullsized":"2024-08-16T19:10:28.309352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308734+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:10:28.308734+0000","last_peered":"2024-08-16T19:10:28.308734+0000","last_clean":"2024-08-16T19:10:28.308734+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:10:28.308734+0000","last_undegraded":"2024-08-16T19:10:28.308734+0000","last_fullsized":"2024-08-16T19:10:28.308734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415737+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:10:28.415737+0000","last_peered":"2024-08-16T19:10:28.415737+0000","last_clean":"2024-08-16T19:10:28.415737+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:10:28.415737+0000","last_undegraded":"2024-08-16T19:10:28.415737+0000","last_fullsized":"2024-08-16T19:10:28.415737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312134+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:10:28.312134+0000","last_peered":"2024-08-16T19:10:28.312134+0000","last_clean":"2024-08-16T19:10:28.312134+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:10:28.312134+0000","last_undegraded":"2024-08-16T19:10:28.312134+0000","last_fullsized":"2024-08-16T19:10:28.312134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405223+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:10:28.405223+0000","last_peered":"2024-08-16T19:10:28.405223+0000","last_clean":"2024-08-16T19:10:28.405223+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:10:28.405223+0000","last_undegraded":"2024-08-16T19:10:28.405223+0000","last_fullsized":"2024-08-16T19:10:28.405223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528437+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:10:28.528437+0000","last_peered":"2024-08-16T19:10:28.528437+0000","last_clean":"2024-08-16T19:10:28.528437+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:10:28.528437+0000","last_undegraded":"2024-08-16T19:10:28.528437+0000","last_fullsized":"2024-08-16T19:10:28.528437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415886+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:10:28.415886+0000","last_peered":"2024-08-16T19:10:28.415886+0000","last_clean":"2024-08-16T19:10:28.415886+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:10:28.415886+0000","last_undegraded":"2024-08-16T19:10:28.415886+0000","last_fullsized":"2024-08-16T19:10:28.415886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416317+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:10:28.416317+0000","last_peered":"2024-08-16T19:10:28.416317+0000","last_clean":"2024-08-16T19:10:28.416317+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:10:28.416317+0000","last_undegraded":"2024-08-16T19:10:28.416317+0000","last_fullsized":"2024-08-16T19:10:28.416317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416497+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:10:28.416497+0000","last_peered":"2024-08-16T19:10:28.416497+0000","last_clean":"2024-08-16T19:10:28.416497+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:10:28.416497+0000","last_undegraded":"2024-08-16T19:10:28.416497+0000","last_fullsized":"2024-08-16T19:10:28.416497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'8","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.585430+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:10:28.585430+0000","last_peered":"2024-08-16T19:10:28.585430+0000","last_clean":"2024-08-16T19:10:28.585430+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:10:28.585430+0000","last_undegraded":"2024-08-16T19:10:28.585430+0000","last_fullsized":"2024-08-16T19:10:28.585430+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2091,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415428+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:10:28.415428+0000","last_peered":"2024-08-16T19:10:28.415428+0000","last_clean":"2024-08-16T19:10:28.415428+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:10:28.415428+0000","last_undegraded":"2024-08-16T19:10:28.415428+0000","last_fullsized":"2024-08-16T19:10:28.415428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416389+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:10:28.416389+0000","last_peered":"2024-08-16T19:10:28.416389+0000","last_clean":"2024-08-16T19:10:28.416389+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:10:28.416389+0000","last_undegraded":"2024-08-16T19:10:28.416389+0000","last_fullsized":"2024-08-16T19:10:28.416389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'41","reported_seq":90,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.524624+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:10:28.524624+0000","last_peered":"2024-08-16T19:10:28.524624+0000","last_clean":"2024-08-16T19:10:28.524624+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:10:28.524624+0000","last_undegraded":"2024-08-16T19:10:28.524624+0000","last_fullsized":"2024-08-16T19:10:28.524624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395202+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:10:28.395202+0000","last_peered":"2024-08-16T19:10:28.395202+0000","last_clean":"2024-08-16T19:10:28.395202+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:10:28.395202+0000","last_undegraded":"2024-08-16T19:10:28.395202+0000","last_fullsized":"2024-08-16T19:10:28.395202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415903+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:10:28.415903+0000","last_peered":"2024-08-16T19:10:28.415903+0000","last_clean":"2024-08-16T19:10:28.415903+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:10:28.415903+0000","last_undegraded":"2024-08-16T19:10:28.415903+0000","last_fullsized":"2024-08-16T19:10:28.415903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309754+0000","last_change":"2024-08-16T19:09:59.590142+0000","last_active":"2024-08-16T19:10:28.309754+0000","last_peered":"2024-08-16T19:10:28.309754+0000","last_clean":"2024-08-16T19:10:28.309754+0000","last_became_active":"2024-08-16T19:09:59.580675+0000","last_became_peered":"2024-08-16T19:09:59.580675+0000","last_unstale":"2024-08-16T19:10:28.309754+0000","last_undegraded":"2024-08-16T19:10:28.309754+0000","last_fullsized":"2024-08-16T19:10:28.309754+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:06:11.306822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311414+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:10:28.311414+0000","last_peered":"2024-08-16T19:10:28.311414+0000","last_clean":"2024-08-16T19:10:28.311414+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:10:28.311414+0000","last_undegraded":"2024-08-16T19:10:28.311414+0000","last_fullsized":"2024-08-16T19:10:28.311414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416700+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:10:28.416700+0000","last_peered":"2024-08-16T19:10:28.416700+0000","last_clean":"2024-08-16T19:10:28.416700+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:10:28.416700+0000","last_undegraded":"2024-08-16T19:10:28.416700+0000","last_fullsized":"2024-08-16T19:10:28.416700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416400+0000","last_change":"2024-08-16T19:09:59.587138+0000","last_active":"2024-08-16T19:10:28.416400+0000","last_peered":"2024-08-16T19:10:28.416400+0000","last_clean":"2024-08-16T19:10:28.416400+0000","last_became_active":"2024-08-16T19:09:59.579143+0000","last_became_peered":"2024-08-16T19:09:59.579143+0000","last_unstale":"2024-08-16T19:10:28.416400+0000","last_undegraded":"2024-08-16T19:10:28.416400+0000","last_fullsized":"2024-08-16T19:10:28.416400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:08.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311491+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:10:28.311491+0000","last_peered":"2024-08-16T19:10:28.311491+0000","last_clean":"2024-08-16T19:10:28.311491+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:10:28.311491+0000","last_undegraded":"2024-08-16T19:10:28.311491+0000","last_fullsized":"2024-08-16T19:10:28.311491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311658+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:10:28.311658+0000","last_peered":"2024-08-16T19:10:28.311658+0000","last_clean":"2024-08-16T19:10:28.311658+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:10:28.311658+0000","last_undegraded":"2024-08-16T19:10:28.311658+0000","last_fullsized":"2024-08-16T19:10:28.311658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404122+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:10:28.404122+0000","last_peered":"2024-08-16T19:10:28.404122+0000","last_clean":"2024-08-16T19:10:28.404122+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:10:28.404122+0000","last_undegraded":"2024-08-16T19:10:28.404122+0000","last_fullsized":"2024-08-16T19:10:28.404122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309655+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:10:28.309655+0000","last_peered":"2024-08-16T19:10:28.309655+0000","last_clean":"2024-08-16T19:10:28.309655+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:10:28.309655+0000","last_undegraded":"2024-08-16T19:10:28.309655+0000","last_fullsized":"2024-08-16T19:10:28.309655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415689+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:10:28.415689+0000","last_peered":"2024-08-16T19:10:28.415689+0000","last_clean":"2024-08-16T19:10:28.415689+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:10:28.415689+0000","last_undegraded":"2024-08-16T19:10:28.415689+0000","last_fullsized":"2024-08-16T19:10:28.415689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":155,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.552691+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:10:28.552691+0000","last_peered":"2024-08-16T19:10:28.552691+0000","last_clean":"2024-08-16T19:10:28.552691+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:10:28.552691+0000","last_undegraded":"2024-08-16T19:10:28.552691+0000","last_fullsized":"2024-08-16T19:10:28.552691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404506+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:10:28.404506+0000","last_peered":"2024-08-16T19:10:28.404506+0000","last_clean":"2024-08-16T19:10:28.404506+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:10:28.404506+0000","last_undegraded":"2024-08-16T19:10:28.404506+0000","last_fullsized":"2024-08-16T19:10:28.404506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415670+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:10:28.415670+0000","last_peered":"2024-08-16T19:10:28.415670+0000","last_clean":"2024-08-16T19:10:28.415670+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:10:28.415670+0000","last_undegraded":"2024-08-16T19:10:28.415670+0000","last_fullsized":"2024-08-16T19:10:28.415670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311811+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:10:28.311811+0000","last_peered":"2024-08-16T19:10:28.311811+0000","last_clean":"2024-08-16T19:10:28.311811+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:10:28.311811+0000","last_undegraded":"2024-08-16T19:10:28.311811+0000","last_fullsized":"2024-08-16T19:10:28.311811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416055+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:10:28.416055+0000","last_peered":"2024-08-16T19:10:28.416055+0000","last_clean":"2024-08-16T19:10:28.416055+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:10:28.416055+0000","last_undegraded":"2024-08-16T19:10:28.416055+0000","last_fullsized":"2024-08-16T19:10:28.416055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308976+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:10:28.308976+0000","last_peered":"2024-08-16T19:10:28.308976+0000","last_clean":"2024-08-16T19:10:28.308976+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:10:28.308976+0000","last_undegraded":"2024-08-16T19:10:28.308976+0000","last_fullsized":"2024-08-16T19:10:28.308976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416259+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:10:28.416259+0000","last_peered":"2024-08-16T19:10:28.416259+0000","last_clean":"2024-08-16T19:10:28.416259+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:10:28.416259+0000","last_undegraded":"2024-08-16T19:10:28.416259+0000","last_fullsized":"2024-08-16T19:10:28.416259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308667+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:10:28.308667+0000","last_peered":"2024-08-16T19:10:28.308667+0000","last_clean":"2024-08-16T19:10:28.308667+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:10:28.308667+0000","last_undegraded":"2024-08-16T19:10:28.308667+0000","last_fullsized":"2024-08-16T19:10:28.308667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404420+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:10:28.404420+0000","last_peered":"2024-08-16T19:10:28.404420+0000","last_clean":"2024-08-16T19:10:28.404420+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:10:28.404420+0000","last_undegraded":"2024-08-16T19:10:28.404420+0000","last_fullsized":"2024-08-16T19:10:28.404420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":298,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.564088+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:10:28.564088+0000","last_peered":"2024-08-16T19:10:28.564088+0000","last_clean":"2024-08-16T19:10:28.564088+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:10:28.564088+0000","last_undegraded":"2024-08-16T19:10:28.564088+0000","last_fullsized":"2024-08-16T19:10:28.564088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.399989+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:10:28.399989+0000","last_peered":"2024-08-16T19:10:28.399989+0000","last_clean":"2024-08-16T19:10:28.399989+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:10:28.399989+0000","last_undegraded":"2024-08-16T19:10:28.399989+0000","last_fullsized":"2024-08-16T19:10:28.399989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415798+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:10:28.415798+0000","last_peered":"2024-08-16T19:10:28.415798+0000","last_clean":"2024-08-16T19:10:28.415798+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:10:28.415798+0000","last_undegraded":"2024-08-16T19:10:28.415798+0000","last_fullsized":"2024-08-16T19:10:28.415798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312030+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:10:28.312030+0000","last_peered":"2024-08-16T19:10:28.312030+0000","last_clean":"2024-08-16T19:10:28.312030+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:10:28.312030+0000","last_undegraded":"2024-08-16T19:10:28.312030+0000","last_fullsized":"2024-08-16T19:10:28.312030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528335+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:10:28.528335+0000","last_peered":"2024-08-16T19:10:28.528335+0000","last_clean":"2024-08-16T19:10:28.528335+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:10:28.528335+0000","last_undegraded":"2024-08-16T19:10:28.528335+0000","last_fullsized":"2024-08-16T19:10:28.528335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395911+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:10:28.395911+0000","last_peered":"2024-08-16T19:10:28.395911+0000","last_clean":"2024-08-16T19:10:28.395911+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:10:28.395911+0000","last_undegraded":"2024-08-16T19:10:28.395911+0000","last_fullsized":"2024-08-16T19:10:28.395911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416395+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:10:28.416395+0000","last_peered":"2024-08-16T19:10:28.416395+0000","last_clean":"2024-08-16T19:10:28.416395+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:10:28.416395+0000","last_undegraded":"2024-08-16T19:10:28.416395+0000","last_fullsized":"2024-08-16T19:10:28.416395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.399854+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:10:28.399854+0000","last_peered":"2024-08-16T19:10:28.399854+0000","last_clean":"2024-08-16T19:10:28.399854+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:10:28.399854+0000","last_undegraded":"2024-08-16T19:10:28.399854+0000","last_fullsized":"2024-08-16T19:10:28.399854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416283+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:10:28.416283+0000","last_peered":"2024-08-16T19:10:28.416283+0000","last_clean":"2024-08-16T19:10:28.416283+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:10:28.416283+0000","last_undegraded":"2024-08-16T19:10:28.416283+0000","last_fullsized":"2024-08-16T19:10:28.416283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":252,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.584792+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:10:28.584792+0000","last_peered":"2024-08-16T19:10:28.584792+0000","last_clean":"2024-08-16T19:10:28.584792+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:10:28.584792+0000","last_undegraded":"2024-08-16T19:10:28.584792+0000","last_fullsized":"2024-08-16T19:10:28.584792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404850+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:10:28.404850+0000","last_peered":"2024-08-16T19:10:28.404850+0000","last_clean":"2024-08-16T19:10:28.404850+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:10:28.404850+0000","last_undegraded":"2024-08-16T19:10:28.404850+0000","last_fullsized":"2024-08-16T19:10:28.404850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395310+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:10:28.395310+0000","last_peered":"2024-08-16T19:10:28.395310+0000","last_clean":"2024-08-16T19:10:28.395310+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:10:28.395310+0000","last_undegraded":"2024-08-16T19:10:28.395310+0000","last_fullsized":"2024-08-16T19:10:28.395310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400012+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:10:28.400012+0000","last_peered":"2024-08-16T19:10:28.400012+0000","last_clean":"2024-08-16T19:10:28.400012+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:10:28.400012+0000","last_undegraded":"2024-08-16T19:10:28.400012+0000","last_fullsized":"2024-08-16T19:10:28.400012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311593+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:10:28.311593+0000","last_peered":"2024-08-16T19:10:28.311593+0000","last_clean":"2024-08-16T19:10:28.311593+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:10:28.311593+0000","last_undegraded":"2024-08-16T19:10:28.311593+0000","last_fullsized":"2024-08-16T19:10:28.311593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415754+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:10:28.415754+0000","last_peered":"2024-08-16T19:10:28.415754+0000","last_clean":"2024-08-16T19:10:28.415754+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:10:28.415754+0000","last_undegraded":"2024-08-16T19:10:28.415754+0000","last_fullsized":"2024-08-16T19:10:28.415754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404608+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:10:28.404608+0000","last_peered":"2024-08-16T19:10:28.404608+0000","last_clean":"2024-08-16T19:10:28.404608+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:10:28.404608+0000","last_undegraded":"2024-08-16T19:10:28.404608+0000","last_fullsized":"2024-08-16T19:10:28.404608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308849+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:10:28.308849+0000","last_peered":"2024-08-16T19:10:28.308849+0000","last_clean":"2024-08-16T19:10:28.308849+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:10:28.308849+0000","last_undegraded":"2024-08-16T19:10:28.308849+0000","last_fullsized":"2024-08-16T19:10:28.308849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308584+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:10:28.308584+0000","last_peered":"2024-08-16T19:10:28.308584+0000","last_clean":"2024-08-16T19:10:28.308584+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:10:28.308584+0000","last_undegraded":"2024-08-16T19:10:28.308584+0000","last_fullsized":"2024-08-16T19:10:28.308584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311961+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:10:28.311961+0000","last_peered":"2024-08-16T19:10:28.311961+0000","last_clean":"2024-08-16T19:10:28.311961+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:10:28.311961+0000","last_undegraded":"2024-08-16T19:10:28.311961+0000","last_fullsized":"2024-08-16T19:10:28.311961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416295+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:10:28.416295+0000","last_peered":"2024-08-16T19:10:28.416295+0000","last_clean":"2024-08-16T19:10:28.416295+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:10:28.416295+0000","last_undegraded":"2024-08-16T19:10:28.416295+0000","last_fullsized":"2024-08-16T19:10:28.416295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.567085+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:10:28.567085+0000","last_peered":"2024-08-16T19:10:28.567085+0000","last_clean":"2024-08-16T19:10:28.567085+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:10:28.567085+0000","last_undegraded":"2024-08-16T19:10:28.567085+0000","last_fullsized":"2024-08-16T19:10:28.567085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404156+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:10:28.404156+0000","last_peered":"2024-08-16T19:10:28.404156+0000","last_clean":"2024-08-16T19:10:28.404156+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:10:28.404156+0000","last_undegraded":"2024-08-16T19:10:28.404156+0000","last_fullsized":"2024-08-16T19:10:28.404156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416359+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:10:28.416359+0000","last_peered":"2024-08-16T19:10:28.416359+0000","last_clean":"2024-08-16T19:10:28.416359+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:10:28.416359+0000","last_undegraded":"2024-08-16T19:10:28.416359+0000","last_fullsized":"2024-08-16T19:10:28.416359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415784+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:10:28.415784+0000","last_peered":"2024-08-16T19:10:28.415784+0000","last_clean":"2024-08-16T19:10:28.415784+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:10:28.415784+0000","last_undegraded":"2024-08-16T19:10:28.415784+0000","last_fullsized":"2024-08-16T19:10:28.415784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415826+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:10:28.415826+0000","last_peered":"2024-08-16T19:10:28.415826+0000","last_clean":"2024-08-16T19:10:28.415826+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:10:28.415826+0000","last_undegraded":"2024-08-16T19:10:28.415826+0000","last_fullsized":"2024-08-16T19:10:28.415826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415629+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:10:28.415629+0000","last_peered":"2024-08-16T19:10:28.415629+0000","last_clean":"2024-08-16T19:10:28.415629+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:10:28.415629+0000","last_undegraded":"2024-08-16T19:10:28.415629+0000","last_fullsized":"2024-08-16T19:10:28.415629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395262+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:10:28.395262+0000","last_peered":"2024-08-16T19:10:28.395262+0000","last_clean":"2024-08-16T19:10:28.395262+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:10:28.395262+0000","last_undegraded":"2024-08-16T19:10:28.395262+0000","last_fullsized":"2024-08-16T19:10:28.395262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309213+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:10:28.309213+0000","last_peered":"2024-08-16T19:10:28.309213+0000","last_clean":"2024-08-16T19:10:28.309213+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:10:28.309213+0000","last_undegraded":"2024-08-16T19:10:28.309213+0000","last_fullsized":"2024-08-16T19:10:28.309213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416309+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:10:28.416309+0000","last_peered":"2024-08-16T19:10:28.416309+0000","last_clean":"2024-08-16T19:10:28.416309+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:10:28.416309+0000","last_undegraded":"2024-08-16T19:10:28.416309+0000","last_fullsized":"2024-08-16T19:10:28.416309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416854+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:10:28.416854+0000","last_peered":"2024-08-16T19:10:28.416854+0000","last_clean":"2024-08-16T19:10:28.416854+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:10:28.416854+0000","last_undegraded":"2024-08-16T19:10:28.416854+0000","last_fullsized":"2024-08-16T19:10:28.416854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406058+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:10:28.406058+0000","last_peered":"2024-08-16T19:10:28.406058+0000","last_clean":"2024-08-16T19:10:28.406058+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:10:28.406058+0000","last_undegraded":"2024-08-16T19:10:28.406058+0000","last_fullsized":"2024-08-16T19:10:28.406058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415471+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:10:28.415471+0000","last_peered":"2024-08-16T19:10:28.415471+0000","last_clean":"2024-08-16T19:10:28.415471+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:10:28.415471+0000","last_undegraded":"2024-08-16T19:10:28.415471+0000","last_fullsized":"2024-08-16T19:10:28.415471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405067+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:10:28.405067+0000","last_peered":"2024-08-16T19:10:28.405067+0000","last_clean":"2024-08-16T19:10:28.405067+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:10:28.405067+0000","last_undegraded":"2024-08-16T19:10:28.405067+0000","last_fullsized":"2024-08-16T19:10:28.405067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415591+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:10:28.415591+0000","last_peered":"2024-08-16T19:10:28.415591+0000","last_clean":"2024-08-16T19:10:28.415591+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:10:28.415591+0000","last_undegraded":"2024-08-16T19:10:28.415591+0000","last_fullsized":"2024-08-16T19:10:28.415591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309143+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:10:28.309143+0000","last_peered":"2024-08-16T19:10:28.309143+0000","last_clean":"2024-08-16T19:10:28.309143+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:10:28.309143+0000","last_undegraded":"2024-08-16T19:10:28.309143+0000","last_fullsized":"2024-08-16T19:10:28.309143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416085+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:10:28.416085+0000","last_peered":"2024-08-16T19:10:28.416085+0000","last_clean":"2024-08-16T19:10:28.416085+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:10:28.416085+0000","last_undegraded":"2024-08-16T19:10:28.416085+0000","last_fullsized":"2024-08-16T19:10:28.416085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416507+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:10:28.416507+0000","last_peered":"2024-08-16T19:10:28.416507+0000","last_clean":"2024-08-16T19:10:28.416507+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:10:28.416507+0000","last_undegraded":"2024-08-16T19:10:28.416507+0000","last_fullsized":"2024-08-16T19:10:28.416507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401991+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:10:28.401991+0000","last_peered":"2024-08-16T19:10:28.401991+0000","last_clean":"2024-08-16T19:10:28.401991+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:10:28.401991+0000","last_undegraded":"2024-08-16T19:10:28.401991+0000","last_fullsized":"2024-08-16T19:10:28.401991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416849+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:10:28.416849+0000","last_peered":"2024-08-16T19:10:28.416849+0000","last_clean":"2024-08-16T19:10:28.416849+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:10:28.416849+0000","last_undegraded":"2024-08-16T19:10:28.416849+0000","last_fullsized":"2024-08-16T19:10:28.416849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415320+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:10:28.415320+0000","last_peered":"2024-08-16T19:10:28.415320+0000","last_clean":"2024-08-16T19:10:28.415320+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:10:28.415320+0000","last_undegraded":"2024-08-16T19:10:28.415320+0000","last_fullsized":"2024-08-16T19:10:28.415320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.394824+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:10:28.394824+0000","last_peered":"2024-08-16T19:10:28.394824+0000","last_clean":"2024-08-16T19:10:28.394824+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:10:28.394824+0000","last_undegraded":"2024-08-16T19:10:28.394824+0000","last_fullsized":"2024-08-16T19:10:28.394824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395082+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:10:28.395082+0000","last_peered":"2024-08-16T19:10:28.395082+0000","last_clean":"2024-08-16T19:10:28.395082+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:10:28.395082+0000","last_undegraded":"2024-08-16T19:10:28.395082+0000","last_fullsized":"2024-08-16T19:10:28.395082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309135+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:10:28.309135+0000","last_peered":"2024-08-16T19:10:28.309135+0000","last_clean":"2024-08-16T19:10:28.309135+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:10:28.309135+0000","last_undegraded":"2024-08-16T19:10:28.309135+0000","last_fullsized":"2024-08-16T19:10:28.309135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528358+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:10:28.528358+0000","last_peered":"2024-08-16T19:10:28.528358+0000","last_clean":"2024-08-16T19:10:28.528358+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:10:28.528358+0000","last_undegraded":"2024-08-16T19:10:28.528358+0000","last_fullsized":"2024-08-16T19:10:28.528358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416564+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:10:28.416564+0000","last_peered":"2024-08-16T19:10:28.416564+0000","last_clean":"2024-08-16T19:10:28.416564+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:10:28.416564+0000","last_undegraded":"2024-08-16T19:10:28.416564+0000","last_fullsized":"2024-08-16T19:10:28.416564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309002+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:10:28.309002+0000","last_peered":"2024-08-16T19:10:28.309002+0000","last_clean":"2024-08-16T19:10:28.309002+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:10:28.309002+0000","last_undegraded":"2024-08-16T19:10:28.309002+0000","last_fullsized":"2024-08-16T19:10:28.309002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416294+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:10:28.416294+0000","last_peered":"2024-08-16T19:10:28.416294+0000","last_clean":"2024-08-16T19:10:28.416294+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:10:28.416294+0000","last_undegraded":"2024-08-16T19:10:28.416294+0000","last_fullsized":"2024-08-16T19:10:28.416294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404467+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:10:28.404467+0000","last_peered":"2024-08-16T19:10:28.404467+0000","last_clean":"2024-08-16T19:10:28.404467+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:10:28.404467+0000","last_undegraded":"2024-08-16T19:10:28.404467+0000","last_fullsized":"2024-08-16T19:10:28.404467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416036+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:10:28.416036+0000","last_peered":"2024-08-16T19:10:28.416036+0000","last_clean":"2024-08-16T19:10:28.416036+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:10:28.416036+0000","last_undegraded":"2024-08-16T19:10:28.416036+0000","last_fullsized":"2024-08-16T19:10:28.416036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308504+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:10:28.308504+0000","last_peered":"2024-08-16T19:10:28.308504+0000","last_clean":"2024-08-16T19:10:28.308504+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:10:28.308504+0000","last_undegraded":"2024-08-16T19:10:28.308504+0000","last_fullsized":"2024-08-16T19:10:28.308504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404114+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:10:28.404114+0000","last_peered":"2024-08-16T19:10:28.404114+0000","last_clean":"2024-08-16T19:10:28.404114+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:10:28.404114+0000","last_undegraded":"2024-08-16T19:10:28.404114+0000","last_fullsized":"2024-08-16T19:10:28.404114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528254+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:10:28.528254+0000","last_peered":"2024-08-16T19:10:28.528254+0000","last_clean":"2024-08-16T19:10:28.528254+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:10:28.528254+0000","last_undegraded":"2024-08-16T19:10:28.528254+0000","last_fullsized":"2024-08-16T19:10:28.528254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395415+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:10:28.395415+0000","last_peered":"2024-08-16T19:10:28.395415+0000","last_clean":"2024-08-16T19:10:28.395415+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:10:28.395415+0000","last_undegraded":"2024-08-16T19:10:28.395415+0000","last_fullsized":"2024-08-16T19:10:28.395415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395414+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:10:28.395414+0000","last_peered":"2024-08-16T19:10:28.395414+0000","last_clean":"2024-08-16T19:10:28.395414+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:10:28.395414+0000","last_undegraded":"2024-08-16T19:10:28.395414+0000","last_fullsized":"2024-08-16T19:10:28.395414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416642+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:10:28.416642+0000","last_peered":"2024-08-16T19:10:28.416642+0000","last_clean":"2024-08-16T19:10:28.416642+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:10:28.416642+0000","last_undegraded":"2024-08-16T19:10:28.416642+0000","last_fullsized":"2024-08-16T19:10:28.416642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603171,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1160,"num_read_kb":694,"num_write":385,"num_write_kb":1476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1809513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":538317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28948,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716204,"statfs":{"total":95995035648,"available":95965392896,"internally_reserved":0,"allocated":1916928,"data_stored":1325101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939699,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1110016,"data_stored":541400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1490944,"data_stored":930876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1136020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":533729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822725,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28816,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716336,"statfs":{"total":95995035648,"available":95965528064,"internally_reserved":0,"allocated":1843200,"data_stored":1260970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":537413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1057332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1171456,"data_stored":597216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":603539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":535185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:10:34.160 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-16T19:10:34.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-16T19:10:34.433 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:34.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:34 smithi062 ceph-mon[32993]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:34.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:34 smithi161 ceph-mon[37576]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:34.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:34 smithi155 ceph-mon[37413]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/1794410407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa01067b0 msgr2=0x7fdfa0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/1794410407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa01067b0 0x7fdfa0106b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009930 tx=0x7fdf9002a420 comp rx=0 tx=0).stop 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/1794410407 shutdown_connections 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/1794410407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa0101150 0x7fdfa010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/1794410407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 0x7fdfa0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/1794410407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa01067b0 0x7fdfa0106b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:34.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/1794410407 >> 172.21.15.62:0/1794410407 conn(0x7fdfa00fc4d0 msgr2=0x7fdfa00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/1794410407 shutdown_connections 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/1794410407 wait complete. 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 Processor -- start 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 -- start start 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 0x7fdfa01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.907+0000 7fdfa6e2f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa6e2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa01067b0 0x7fdfa019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa6e2f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0111cd0 con 0x7fdfa0101150 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa6e2f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0111b50 con 0x7fdfa01067b0 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa6e2f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0111e50 con 0x7fdfa01007b0 2024-08-16T19:10:34.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44310/0 (socket says 172.21.15.62:44310) 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 -- 172.21.15.62:0/3145075148 learned_addr learned my addr 172.21.15.62:0/3145075148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa4bcb640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 0x7fdfa01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa53cc640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa01067b0 0x7fdfa019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 -- 172.21.15.62:0/3145075148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 msgr2=0x7fdfa01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 0x7fdfa01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 -- 172.21.15.62:0/3145075148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa01067b0 msgr2=0x7fdfa019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa01067b0 0x7fdfa019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 -- 172.21.15.62:0/3145075148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa019d1a0 con 0x7fdfa0101150 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9ffff640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9f0 tx=0x7fdf8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800cd80 con 0x7fdfa0101150 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88005500 con 0x7fdfa0101150 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa019d490 con 0x7fdfa0101150 2024-08-16T19:10:34.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.908+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88010690 con 0x7fdfa0101150 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.909+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa01a4d70 con 0x7fdfa0101150 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.910+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0100c10 con 0x7fdfa0101150 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.911+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fdf880046c0 con 0x7fdfa0101150 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.912+0000 7fdf9dffb640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 0x7fdf8c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.912+0000 7fdfa4bcb640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 0x7fdf8c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.912+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fdf880cfdb0 con 0x7fdfa0101150 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.912+0000 7fdfa4bcb640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 0x7fdf8c0a52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009900 tx=0x7fdf900058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:34.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:34.916+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdf8809c4d0 con 0x7fdfa0101150 2024-08-16T19:10:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.088+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfa0062360 con 0x7fdf8c0a2e00 2024-08-16T19:10:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.098+0000 7fdf9dffb640 1 -- 172.21.15.62:0/3145075148 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339461 (secure 0 0 0) 0x7fdfa0062360 con 0x7fdf8c0a2e00 2024-08-16T19:10:35.102 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 msgr2=0x7fdf8c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 0x7fdf8c0a52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009900 tx=0x7fdf900058d0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 msgr2=0x7fdfa0198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800e9f0 tx=0x7fdf8800edf0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 shutdown_connections 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fdf8c0a2e00 0x7fdf8c0a52c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfa01067b0 0x7fdfa019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfa0101150 0x7fdfa0198730 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 --2- 172.21.15.62:0/3145075148 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfa01007b0 0x7fdfa01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 >> 172.21.15.62:0/3145075148 conn(0x7fdfa00fc4d0 msgr2=0x7fdfa00fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:35.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 shutdown_connections 2024-08-16T19:10:35.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:35.102+0000 7fdfa6e2f640 1 -- 172.21.15.62:0/3145075148 wait complete. 2024-08-16T19:10:35.412 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:35 smithi062 ceph-mon[32993]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 80 op/s 2024-08-16T19:10:35.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:35 smithi161 ceph-mon[37576]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 80 op/s 2024-08-16T19:10:35.778 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-08-16T19:10:34.322355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390405,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1570,"num_read_kb":1049,"num_write":694,"num_write_kb":5330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1028,"ondisk_log_size":1028,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382884,"kb_used_data":16760,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124558940,"statfs":{"total":1151940427776,"available":1151548354560,"internally_reserved":0,"allocated":17162240,"data_stored":10391681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":274,"num_read_kb":236,"num_write":126,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990749"},"pg_stats":[{"pgid":"2.1f","version":"93'15","reported_seq":163,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:32.564122+0000","last_change":"2024-08-16T19:08:02.453151+0000","last_active":"2024-08-16T19:10:32.564122+0000","last_peered":"2024-08-16T19:10:32.564122+0000","last_clean":"2024-08-16T19:10:32.564122+0000","last_became_active":"2024-08-16T19:08:02.452779+0000","last_became_peered":"2024-08-16T19:08:02.452779+0000","last_unstale":"2024-08-16T19:10:32.564122+0000","last_undegraded":"2024-08-16T19:10:32.564122+0000","last_fullsized":"2024-08-16T19:10:32.564122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:51:38.023093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406289+0000","last_change":"2024-08-16T19:08:04.593030+0000","last_active":"2024-08-16T19:10:28.406289+0000","last_peered":"2024-08-16T19:10:28.406289+0000","last_clean":"2024-08-16T19:10:28.406289+0000","last_became_active":"2024-08-16T19:08:04.592472+0000","last_became_peered":"2024-08-16T19:08:04.592472+0000","last_unstale":"2024-08-16T19:10:28.406289+0000","last_undegraded":"2024-08-16T19:10:28.406289+0000","last_fullsized":"2024-08-16T19:10:28.406289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:10:33.986213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415552+0000","last_change":"2024-08-16T19:08:11.630971+0000","last_active":"2024-08-16T19:10:28.415552+0000","last_peered":"2024-08-16T19:10:28.415552+0000","last_clean":"2024-08-16T19:10:28.415552+0000","last_became_active":"2024-08-16T19:08:11.630592+0000","last_became_peered":"2024-08-16T19:08:11.630592+0000","last_unstale":"2024-08-16T19:10:28.415552+0000","last_undegraded":"2024-08-16T19:10:28.415552+0000","last_fullsized":"2024-08-16T19:10:28.415552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:43:22.807814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406640+0000","last_change":"2024-08-16T19:08:02.976111+0000","last_active":"2024-08-16T19:10:28.406640+0000","last_peered":"2024-08-16T19:10:28.406640+0000","last_clean":"2024-08-16T19:10:28.406640+0000","last_became_active":"2024-08-16T19:08:02.975759+0000","last_became_peered":"2024-08-16T19:08:02.975759+0000","last_unstale":"2024-08-16T19:10:28.406640+0000","last_undegraded":"2024-08-16T19:10:28.406640+0000","last_fullsized":"2024-08-16T19:10:28.406640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:14:38.941259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395855+0000","last_change":"2024-08-16T19:08:04.591453+0000","last_active":"2024-08-16T19:10:28.395855+0000","last_peered":"2024-08-16T19:10:28.395855+0000","last_clean":"2024-08-16T19:10:28.395855+0000","last_became_active":"2024-08-16T19:08:04.591171+0000","last_became_peered":"2024-08-16T19:08:04.591171+0000","last_unstale":"2024-08-16T19:10:28.395855+0000","last_undegraded":"2024-08-16T19:10:28.395855+0000","last_fullsized":"2024-08-16T19:10:28.395855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:41:26.896168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527883+0000","last_change":"2024-08-16T19:08:59.594556+0000","last_active":"2024-08-16T19:10:28.527883+0000","last_peered":"2024-08-16T19:10:28.527883+0000","last_clean":"2024-08-16T19:10:28.527883+0000","last_became_active":"2024-08-16T19:08:59.594115+0000","last_became_peered":"2024-08-16T19:08:59.594115+0000","last_unstale":"2024-08-16T19:10:28.527883+0000","last_undegraded":"2024-08-16T19:10:28.527883+0000","last_fullsized":"2024-08-16T19:10:28.527883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:04.932914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404475+0000","last_change":"2024-08-16T19:08:02.459391+0000","last_active":"2024-08-16T19:10:28.404475+0000","last_peered":"2024-08-16T19:10:28.404475+0000","last_clean":"2024-08-16T19:10:28.404475+0000","last_became_active":"2024-08-16T19:08:02.458897+0000","last_became_peered":"2024-08-16T19:08:02.458897+0000","last_unstale":"2024-08-16T19:10:28.404475+0000","last_undegraded":"2024-08-16T19:10:28.404475+0000","last_fullsized":"2024-08-16T19:10:28.404475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:00:28.437433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415713+0000","last_change":"2024-08-16T19:08:04.597273+0000","last_active":"2024-08-16T19:10:28.415713+0000","last_peered":"2024-08-16T19:10:28.415713+0000","last_clean":"2024-08-16T19:10:28.415713+0000","last_became_active":"2024-08-16T19:08:04.596867+0000","last_became_peered":"2024-08-16T19:08:04.596867+0000","last_unstale":"2024-08-16T19:10:28.415713+0000","last_undegraded":"2024-08-16T19:10:28.415713+0000","last_fullsized":"2024-08-16T19:10:28.415713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:46:54.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528080+0000","last_change":"2024-08-16T19:08:11.630922+0000","last_active":"2024-08-16T19:10:28.528080+0000","last_peered":"2024-08-16T19:10:28.528080+0000","last_clean":"2024-08-16T19:10:28.528080+0000","last_became_active":"2024-08-16T19:08:11.630306+0000","last_became_peered":"2024-08-16T19:08:11.630306+0000","last_unstale":"2024-08-16T19:10:28.528080+0000","last_undegraded":"2024-08-16T19:10:28.528080+0000","last_fullsized":"2024-08-16T19:10:28.528080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:11.560134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404392+0000","last_change":"2024-08-16T19:08:02.458972+0000","last_active":"2024-08-16T19:10:28.404392+0000","last_peered":"2024-08-16T19:10:28.404392+0000","last_clean":"2024-08-16T19:10:28.404392+0000","last_became_active":"2024-08-16T19:08:02.458535+0000","last_became_peered":"2024-08-16T19:08:02.458535+0000","last_unstale":"2024-08-16T19:10:28.404392+0000","last_undegraded":"2024-08-16T19:10:28.404392+0000","last_fullsized":"2024-08-16T19:10:28.404392+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:26:41.811257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400420+0000","last_change":"2024-08-16T19:08:59.547457+0000","last_active":"2024-08-16T19:10:28.400420+0000","last_peered":"2024-08-16T19:10:28.400420+0000","last_clean":"2024-08-16T19:10:28.400420+0000","last_became_active":"2024-08-16T19:08:59.547063+0000","last_became_peered":"2024-08-16T19:08:59.547063+0000","last_unstale":"2024-08-16T19:10:28.400420+0000","last_undegraded":"2024-08-16T19:10:28.400420+0000","last_fullsized":"2024-08-16T19:10:28.400420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:02:35.708872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.527974+0000","last_change":"2024-08-16T19:08:11.630284+0000","last_active":"2024-08-16T19:10:28.527974+0000","last_peered":"2024-08-16T19:10:28.527974+0000","last_clean":"2024-08-16T19:10:28.527974+0000","last_became_active":"2024-08-16T19:08:11.629873+0000","last_became_peered":"2024-08-16T19:08:11.629873+0000","last_unstale":"2024-08-16T19:10:28.527974+0000","last_undegraded":"2024-08-16T19:10:28.527974+0000","last_fullsized":"2024-08-16T19:10:28.527974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:33:05.555702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416184+0000","last_change":"2024-08-16T19:08:04.603396+0000","last_active":"2024-08-16T19:10:28.416184+0000","last_peered":"2024-08-16T19:10:28.416184+0000","last_clean":"2024-08-16T19:10:28.416184+0000","last_became_active":"2024-08-16T19:08:04.602773+0000","last_became_peered":"2024-08-16T19:08:04.602773+0000","last_unstale":"2024-08-16T19:10:28.416184+0000","last_undegraded":"2024-08-16T19:10:28.416184+0000","last_fullsized":"2024-08-16T19:10:28.416184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:47:17.092278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404284+0000","last_change":"2024-08-16T19:08:02.461179+0000","last_active":"2024-08-16T19:10:28.404284+0000","last_peered":"2024-08-16T19:10:28.404284+0000","last_clean":"2024-08-16T19:10:28.404284+0000","last_became_active":"2024-08-16T19:08:02.460830+0000","last_became_peered":"2024-08-16T19:08:02.460830+0000","last_unstale":"2024-08-16T19:10:28.404284+0000","last_undegraded":"2024-08-16T19:10:28.404284+0000","last_fullsized":"2024-08-16T19:10:28.404284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:55:46.725712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528162+0000","last_change":"2024-08-16T19:08:11.677904+0000","last_active":"2024-08-16T19:10:28.528162+0000","last_peered":"2024-08-16T19:10:28.528162+0000","last_clean":"2024-08-16T19:10:28.528162+0000","last_became_active":"2024-08-16T19:08:11.677439+0000","last_became_peered":"2024-08-16T19:08:11.677439+0000","last_unstale":"2024-08-16T19:10:28.528162+0000","last_undegraded":"2024-08-16T19:10:28.528162+0000","last_fullsized":"2024-08-16T19:10:28.528162+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:33:36.251177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404992+0000","last_change":"2024-08-16T19:08:04.591217+0000","last_active":"2024-08-16T19:10:28.404992+0000","last_peered":"2024-08-16T19:10:28.404992+0000","last_clean":"2024-08-16T19:10:28.404992+0000","last_became_active":"2024-08-16T19:08:04.590924+0000","last_became_peered":"2024-08-16T19:08:04.590924+0000","last_unstale":"2024-08-16T19:10:28.404992+0000","last_undegraded":"2024-08-16T19:10:28.404992+0000","last_fullsized":"2024-08-16T19:10:28.404992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:27.831947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416087+0000","last_change":"2024-08-16T19:09:59.583331+0000","last_active":"2024-08-16T19:10:28.416087+0000","last_peered":"2024-08-16T19:10:28.416087+0000","last_clean":"2024-08-16T19:10:28.416087+0000","last_became_active":"2024-08-16T19:09:59.578673+0000","last_became_peered":"2024-08-16T19:09:59.578673+0000","last_unstale":"2024-08-16T19:10:28.416087+0000","last_undegraded":"2024-08-16T19:10:28.416087+0000","last_fullsized":"2024-08-16T19:10:28.416087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:00:56.841049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528119+0000","last_change":"2024-08-16T19:08:11.627506+0000","last_active":"2024-08-16T19:10:28.528119+0000","last_peered":"2024-08-16T19:10:28.528119+0000","last_clean":"2024-08-16T19:10:28.528119+0000","last_became_active":"2024-08-16T19:08:11.627080+0000","last_became_peered":"2024-08-16T19:08:11.627080+0000","last_unstale":"2024-08-16T19:10:28.528119+0000","last_undegraded":"2024-08-16T19:10:28.528119+0000","last_fullsized":"2024-08-16T19:10:28.528119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:37:27.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415835+0000","last_change":"2024-08-16T19:08:04.591461+0000","last_active":"2024-08-16T19:10:28.415835+0000","last_peered":"2024-08-16T19:10:28.415835+0000","last_clean":"2024-08-16T19:10:28.415835+0000","last_became_active":"2024-08-16T19:08:04.591145+0000","last_became_peered":"2024-08-16T19:08:04.591145+0000","last_unstale":"2024-08-16T19:10:28.415835+0000","last_undegraded":"2024-08-16T19:10:28.415835+0000","last_fullsized":"2024-08-16T19:10:28.415835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:11:31.465854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405227+0000","last_change":"2024-08-16T19:08:02.452184+0000","last_active":"2024-08-16T19:10:28.405227+0000","last_peered":"2024-08-16T19:10:28.405227+0000","last_clean":"2024-08-16T19:10:28.405227+0000","last_became_active":"2024-08-16T19:08:02.451852+0000","last_became_peered":"2024-08-16T19:08:02.451852+0000","last_unstale":"2024-08-16T19:10:28.405227+0000","last_undegraded":"2024-08-16T19:10:28.405227+0000","last_fullsized":"2024-08-16T19:10:28.405227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:56:48.650474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401033+0000","last_change":"2024-08-16T19:08:11.635211+0000","last_active":"2024-08-16T19:10:28.401033+0000","last_peered":"2024-08-16T19:10:28.401033+0000","last_clean":"2024-08-16T19:10:28.401033+0000","last_became_active":"2024-08-16T19:08:11.633282+0000","last_became_peered":"2024-08-16T19:08:11.633282+0000","last_unstale":"2024-08-16T19:10:28.401033+0000","last_undegraded":"2024-08-16T19:10:28.401033+0000","last_fullsized":"2024-08-16T19:10:28.401033+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:18.161587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404046+0000","last_change":"2024-08-16T19:08:04.586549+0000","last_active":"2024-08-16T19:10:28.404046+0000","last_peered":"2024-08-16T19:10:28.404046+0000","last_clean":"2024-08-16T19:10:28.404046+0000","last_became_active":"2024-08-16T19:08:04.586216+0000","last_became_peered":"2024-08-16T19:08:04.586216+0000","last_unstale":"2024-08-16T19:10:28.404046+0000","last_undegraded":"2024-08-16T19:10:28.404046+0000","last_fullsized":"2024-08-16T19:10:28.404046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:46:56.958777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415390+0000","last_change":"2024-08-16T19:08:02.975951+0000","last_active":"2024-08-16T19:10:28.415390+0000","last_peered":"2024-08-16T19:10:28.415390+0000","last_clean":"2024-08-16T19:10:28.415390+0000","last_became_active":"2024-08-16T19:08:02.975644+0000","last_became_peered":"2024-08-16T19:08:02.975644+0000","last_unstale":"2024-08-16T19:10:28.415390+0000","last_undegraded":"2024-08-16T19:10:28.415390+0000","last_fullsized":"2024-08-16T19:10:28.415390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:47:27.166353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528239+0000","last_change":"2024-08-16T19:08:59.464030+0000","last_active":"2024-08-16T19:10:28.528239+0000","last_peered":"2024-08-16T19:10:28.528239+0000","last_clean":"2024-08-16T19:10:28.528239+0000","last_became_active":"2024-08-16T19:08:59.463596+0000","last_became_peered":"2024-08-16T19:08:59.463596+0000","last_unstale":"2024-08-16T19:10:28.528239+0000","last_undegraded":"2024-08-16T19:10:28.528239+0000","last_fullsized":"2024-08-16T19:10:28.528239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:29:54.873031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395532+0000","last_change":"2024-08-16T19:08:04.595458+0000","last_active":"2024-08-16T19:10:28.395532+0000","last_peered":"2024-08-16T19:10:28.395532+0000","last_clean":"2024-08-16T19:10:28.395532+0000","last_became_active":"2024-08-16T19:08:04.595121+0000","last_became_peered":"2024-08-16T19:08:04.595121+0000","last_unstale":"2024-08-16T19:10:28.395532+0000","last_undegraded":"2024-08-16T19:10:28.395532+0000","last_fullsized":"2024-08-16T19:10:28.395532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:56:59.563766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528368+0000","last_change":"2024-08-16T19:08:02.621868+0000","last_active":"2024-08-16T19:10:28.528368+0000","last_peered":"2024-08-16T19:10:28.528368+0000","last_clean":"2024-08-16T19:10:28.528368+0000","last_became_active":"2024-08-16T19:08:02.621478+0000","last_became_peered":"2024-08-16T19:08:02.621478+0000","last_unstale":"2024-08-16T19:10:28.528368+0000","last_undegraded":"2024-08-16T19:10:28.528368+0000","last_fullsized":"2024-08-16T19:10:28.528368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:32.024814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415553+0000","last_change":"2024-08-16T19:08:11.624386+0000","last_active":"2024-08-16T19:10:28.415553+0000","last_peered":"2024-08-16T19:10:28.415553+0000","last_clean":"2024-08-16T19:10:28.415553+0000","last_became_active":"2024-08-16T19:08:11.624035+0000","last_became_peered":"2024-08-16T19:08:11.624035+0000","last_unstale":"2024-08-16T19:10:28.415553+0000","last_undegraded":"2024-08-16T19:10:28.415553+0000","last_fullsized":"2024-08-16T19:10:28.415553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:25:41.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400776+0000","last_change":"2024-08-16T19:08:59.463105+0000","last_active":"2024-08-16T19:10:28.400776+0000","last_peered":"2024-08-16T19:10:28.400776+0000","last_clean":"2024-08-16T19:10:28.400776+0000","last_became_active":"2024-08-16T19:08:59.462797+0000","last_became_peered":"2024-08-16T19:08:59.462797+0000","last_unstale":"2024-08-16T19:10:28.400776+0000","last_undegraded":"2024-08-16T19:10:28.400776+0000","last_fullsized":"2024-08-16T19:10:28.400776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:06:40.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"93'105","reported_seq":238,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:30.560200+0000","last_change":"2024-08-16T19:08:02.452098+0000","last_active":"2024-08-16T19:10:30.560200+0000","last_peered":"2024-08-16T19:10:30.560200+0000","last_clean":"2024-08-16T19:10:30.560200+0000","last_became_active":"2024-08-16T19:08:02.451769+0000","last_became_peered":"2024-08-16T19:08:02.451769+0000","last_unstale":"2024-08-16T19:10:30.560200+0000","last_undegraded":"2024-08-16T19:10:30.560200+0000","last_fullsized":"2024-08-16T19:10:30.560200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:28:06.218320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":192,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404310+0000","last_change":"2024-08-16T19:08:11.678048+0000","last_active":"2024-08-16T19:10:28.404310+0000","last_peered":"2024-08-16T19:10:28.404310+0000","last_clean":"2024-08-16T19:10:28.404310+0000","last_became_active":"2024-08-16T19:08:11.677618+0000","last_became_peered":"2024-08-16T19:08:11.677618+0000","last_unstale":"2024-08-16T19:10:28.404310+0000","last_undegraded":"2024-08-16T19:10:28.404310+0000","last_fullsized":"2024-08-16T19:10:28.404310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:33:48.194118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309239+0000","last_change":"2024-08-16T19:08:04.592490+0000","last_active":"2024-08-16T19:10:28.309239+0000","last_peered":"2024-08-16T19:10:28.309239+0000","last_clean":"2024-08-16T19:10:28.309239+0000","last_became_active":"2024-08-16T19:08:04.592140+0000","last_became_peered":"2024-08-16T19:08:04.592140+0000","last_unstale":"2024-08-16T19:10:28.309239+0000","last_undegraded":"2024-08-16T19:10:28.309239+0000","last_fullsized":"2024-08-16T19:10:28.309239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:08:25.886447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404515+0000","last_change":"2024-08-16T19:08:02.978212+0000","last_active":"2024-08-16T19:10:28.404515+0000","last_peered":"2024-08-16T19:10:28.404515+0000","last_clean":"2024-08-16T19:10:28.404515+0000","last_became_active":"2024-08-16T19:08:02.977771+0000","last_became_peered":"2024-08-16T19:08:02.977771+0000","last_unstale":"2024-08-16T19:10:28.404515+0000","last_undegraded":"2024-08-16T19:10:28.404515+0000","last_fullsized":"2024-08-16T19:10:28.404515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:21:36.239378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416028+0000","last_change":"2024-08-16T19:08:11.628636+0000","last_active":"2024-08-16T19:10:28.416028+0000","last_peered":"2024-08-16T19:10:28.416028+0000","last_clean":"2024-08-16T19:10:28.416028+0000","last_became_active":"2024-08-16T19:08:11.628251+0000","last_became_peered":"2024-08-16T19:08:11.628251+0000","last_unstale":"2024-08-16T19:10:28.416028+0000","last_undegraded":"2024-08-16T19:10:28.416028+0000","last_fullsized":"2024-08-16T19:10:28.416028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:31:18.796017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308802+0000","last_change":"2024-08-16T19:08:04.592641+0000","last_active":"2024-08-16T19:10:28.308802+0000","last_peered":"2024-08-16T19:10:28.308802+0000","last_clean":"2024-08-16T19:10:28.308802+0000","last_became_active":"2024-08-16T19:08:04.591697+0000","last_became_peered":"2024-08-16T19:08:04.591697+0000","last_unstale":"2024-08-16T19:10:28.308802+0000","last_undegraded":"2024-08-16T19:10:28.308802+0000","last_fullsized":"2024-08-16T19:10:28.308802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:37:22.593641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401633+0000","last_change":"2024-08-16T19:08:02.977631+0000","last_active":"2024-08-16T19:10:28.401633+0000","last_peered":"2024-08-16T19:10:28.401633+0000","last_clean":"2024-08-16T19:10:28.401633+0000","last_became_active":"2024-08-16T19:08:02.977302+0000","last_became_peered":"2024-08-16T19:08:02.977302+0000","last_unstale":"2024-08-16T19:10:28.401633+0000","last_undegraded":"2024-08-16T19:10:28.401633+0000","last_fullsized":"2024-08-16T19:10:28.401633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:03:00.241729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404063+0000","last_change":"2024-08-16T19:08:11.679294+0000","last_active":"2024-08-16T19:10:28.404063+0000","last_peered":"2024-08-16T19:10:28.404063+0000","last_clean":"2024-08-16T19:10:28.404063+0000","last_became_active":"2024-08-16T19:08:11.678893+0000","last_became_peered":"2024-08-16T19:08:11.678893+0000","last_unstale":"2024-08-16T19:10:28.404063+0000","last_undegraded":"2024-08-16T19:10:28.404063+0000","last_fullsized":"2024-08-16T19:10:28.404063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:12:18.811280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308652+0000","last_change":"2024-08-16T19:08:04.595296+0000","last_active":"2024-08-16T19:10:28.308652+0000","last_peered":"2024-08-16T19:10:28.308652+0000","last_clean":"2024-08-16T19:10:28.308652+0000","last_became_active":"2024-08-16T19:08:04.594943+0000","last_became_peered":"2024-08-16T19:08:04.594943+0000","last_unstale":"2024-08-16T19:10:28.308652+0000","last_undegraded":"2024-08-16T19:10:28.308652+0000","last_fullsized":"2024-08-16T19:10:28.308652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:17:40.182665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"93'8","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.559614+0000","last_change":"2024-08-16T19:08:02.455813+0000","last_active":"2024-08-16T19:10:28.559614+0000","last_peered":"2024-08-16T19:10:28.559614+0000","last_clean":"2024-08-16T19:10:28.559614+0000","last_became_active":"2024-08-16T19:08:02.455392+0000","last_became_peered":"2024-08-16T19:08:02.455392+0000","last_unstale":"2024-08-16T19:10:28.559614+0000","last_undegraded":"2024-08-16T19:10:28.559614+0000","last_fullsized":"2024-08-16T19:10:28.559614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:52:47.716625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1509,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309099+0000","last_change":"2024-08-16T19:08:11.629850+0000","last_active":"2024-08-16T19:10:28.309099+0000","last_peered":"2024-08-16T19:10:28.309099+0000","last_clean":"2024-08-16T19:10:28.309099+0000","last_became_active":"2024-08-16T19:08:11.629457+0000","last_became_peered":"2024-08-16T19:08:11.629457+0000","last_unstale":"2024-08-16T19:10:28.309099+0000","last_undegraded":"2024-08-16T19:10:28.309099+0000","last_fullsized":"2024-08-16T19:10:28.309099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:23:37.769222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311486+0000","last_change":"2024-08-16T19:08:04.591145+0000","last_active":"2024-08-16T19:10:28.311486+0000","last_peered":"2024-08-16T19:10:28.311486+0000","last_clean":"2024-08-16T19:10:28.311486+0000","last_became_active":"2024-08-16T19:08:04.590656+0000","last_became_peered":"2024-08-16T19:08:04.590656+0000","last_unstale":"2024-08-16T19:10:28.311486+0000","last_undegraded":"2024-08-16T19:10:28.311486+0000","last_fullsized":"2024-08-16T19:10:28.311486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:11:35.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308957+0000","last_change":"2024-08-16T19:08:02.976094+0000","last_active":"2024-08-16T19:10:28.308957+0000","last_peered":"2024-08-16T19:10:28.308957+0000","last_clean":"2024-08-16T19:10:28.308957+0000","last_became_active":"2024-08-16T19:08:02.975696+0000","last_became_peered":"2024-08-16T19:08:02.975696+0000","last_unstale":"2024-08-16T19:10:28.308957+0000","last_undegraded":"2024-08-16T19:10:28.308957+0000","last_fullsized":"2024-08-16T19:10:28.308957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:58:18.841033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309352+0000","last_change":"2024-08-16T19:08:11.678623+0000","last_active":"2024-08-16T19:10:28.309352+0000","last_peered":"2024-08-16T19:10:28.309352+0000","last_clean":"2024-08-16T19:10:28.309352+0000","last_became_active":"2024-08-16T19:08:11.678030+0000","last_became_peered":"2024-08-16T19:08:11.678030+0000","last_unstale":"2024-08-16T19:10:28.309352+0000","last_undegraded":"2024-08-16T19:10:28.309352+0000","last_fullsized":"2024-08-16T19:10:28.309352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:39:47.140611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308734+0000","last_change":"2024-08-16T19:08:04.599027+0000","last_active":"2024-08-16T19:10:28.308734+0000","last_peered":"2024-08-16T19:10:28.308734+0000","last_clean":"2024-08-16T19:10:28.308734+0000","last_became_active":"2024-08-16T19:08:04.598661+0000","last_became_peered":"2024-08-16T19:08:04.598661+0000","last_unstale":"2024-08-16T19:10:28.308734+0000","last_undegraded":"2024-08-16T19:10:28.308734+0000","last_fullsized":"2024-08-16T19:10:28.308734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:37.576809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415737+0000","last_change":"2024-08-16T19:08:02.457112+0000","last_active":"2024-08-16T19:10:28.415737+0000","last_peered":"2024-08-16T19:10:28.415737+0000","last_clean":"2024-08-16T19:10:28.415737+0000","last_became_active":"2024-08-16T19:08:02.456706+0000","last_became_peered":"2024-08-16T19:08:02.456706+0000","last_unstale":"2024-08-16T19:10:28.415737+0000","last_undegraded":"2024-08-16T19:10:28.415737+0000","last_fullsized":"2024-08-16T19:10:28.415737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:14:02.206827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312134+0000","last_change":"2024-08-16T19:08:11.679344+0000","last_active":"2024-08-16T19:10:28.312134+0000","last_peered":"2024-08-16T19:10:28.312134+0000","last_clean":"2024-08-16T19:10:28.312134+0000","last_became_active":"2024-08-16T19:08:11.678909+0000","last_became_peered":"2024-08-16T19:08:11.678909+0000","last_unstale":"2024-08-16T19:10:28.312134+0000","last_undegraded":"2024-08-16T19:10:28.312134+0000","last_fullsized":"2024-08-16T19:10:28.312134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:10:33.564116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405223+0000","last_change":"2024-08-16T19:08:04.596369+0000","last_active":"2024-08-16T19:10:28.405223+0000","last_peered":"2024-08-16T19:10:28.405223+0000","last_clean":"2024-08-16T19:10:28.405223+0000","last_became_active":"2024-08-16T19:08:04.596070+0000","last_became_peered":"2024-08-16T19:08:04.596070+0000","last_unstale":"2024-08-16T19:10:28.405223+0000","last_undegraded":"2024-08-16T19:10:28.405223+0000","last_fullsized":"2024-08-16T19:10:28.405223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:45.998403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528437+0000","last_change":"2024-08-16T19:08:02.452682+0000","last_active":"2024-08-16T19:10:28.528437+0000","last_peered":"2024-08-16T19:10:28.528437+0000","last_clean":"2024-08-16T19:10:28.528437+0000","last_became_active":"2024-08-16T19:08:02.452345+0000","last_became_peered":"2024-08-16T19:08:02.452345+0000","last_unstale":"2024-08-16T19:10:28.528437+0000","last_undegraded":"2024-08-16T19:10:28.528437+0000","last_fullsized":"2024-08-16T19:10:28.528437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:39:39.268995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415886+0000","last_change":"2024-08-16T19:06:55.149067+0000","last_active":"2024-08-16T19:10:28.415886+0000","last_peered":"2024-08-16T19:10:28.415886+0000","last_clean":"2024-08-16T19:10:28.415886+0000","last_became_active":"2024-08-16T19:06:55.148668+0000","last_became_peered":"2024-08-16T19:06:55.148668+0000","last_unstale":"2024-08-16T19:10:28.415886+0000","last_undegraded":"2024-08-16T19:10:28.415886+0000","last_fullsized":"2024-08-16T19:10:28.415886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:03:58.216224+0000","last_clean_scrub_stamp":"2024-08-16T19:03:58.216224+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:59:40.714987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416317+0000","last_change":"2024-08-16T19:08:11.627065+0000","last_active":"2024-08-16T19:10:28.416317+0000","last_peered":"2024-08-16T19:10:28.416317+0000","last_clean":"2024-08-16T19:10:28.416317+0000","last_became_active":"2024-08-16T19:08:11.626604+0000","last_became_peered":"2024-08-16T19:08:11.626604+0000","last_unstale":"2024-08-16T19:10:28.416317+0000","last_undegraded":"2024-08-16T19:10:28.416317+0000","last_fullsized":"2024-08-16T19:10:28.416317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:05:28.091537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416497+0000","last_change":"2024-08-16T19:08:04.604398+0000","last_active":"2024-08-16T19:10:28.416497+0000","last_peered":"2024-08-16T19:10:28.416497+0000","last_clean":"2024-08-16T19:10:28.416497+0000","last_became_active":"2024-08-16T19:08:04.602441+0000","last_became_peered":"2024-08-16T19:08:04.602441+0000","last_unstale":"2024-08-16T19:10:28.416497+0000","last_undegraded":"2024-08-16T19:10:28.416497+0000","last_fullsized":"2024-08-16T19:10:28.416497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:38:21.700675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"93'8","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:32.591021+0000","last_change":"2024-08-16T19:08:02.976476+0000","last_active":"2024-08-16T19:10:32.591021+0000","last_peered":"2024-08-16T19:10:32.591021+0000","last_clean":"2024-08-16T19:10:32.591021+0000","last_became_active":"2024-08-16T19:08:02.975965+0000","last_became_peered":"2024-08-16T19:08:02.975965+0000","last_unstale":"2024-08-16T19:10:32.591021+0000","last_undegraded":"2024-08-16T19:10:32.591021+0000","last_fullsized":"2024-08-16T19:10:32.591021+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:12:58.105395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2091,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415428+0000","last_change":"2024-08-16T19:08:11.627774+0000","last_active":"2024-08-16T19:10:28.415428+0000","last_peered":"2024-08-16T19:10:28.415428+0000","last_clean":"2024-08-16T19:10:28.415428+0000","last_became_active":"2024-08-16T19:08:11.627350+0000","last_became_peered":"2024-08-16T19:08:11.627350+0000","last_unstale":"2024-08-16T19:10:28.415428+0000","last_undegraded":"2024-08-16T19:10:28.415428+0000","last_fullsized":"2024-08-16T19:10:28.415428+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:37:24.699675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416389+0000","last_change":"2024-08-16T19:08:04.601655+0000","last_active":"2024-08-16T19:10:28.416389+0000","last_peered":"2024-08-16T19:10:28.416389+0000","last_clean":"2024-08-16T19:10:28.416389+0000","last_became_active":"2024-08-16T19:08:04.601335+0000","last_became_peered":"2024-08-16T19:08:04.601335+0000","last_unstale":"2024-08-16T19:10:28.416389+0000","last_undegraded":"2024-08-16T19:10:28.416389+0000","last_fullsized":"2024-08-16T19:10:28.416389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:19:39.159871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'67","reported_seq":139,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:30.152356+0000","last_change":"2024-08-16T19:08:02.458417+0000","last_active":"2024-08-16T19:10:30.152356+0000","last_peered":"2024-08-16T19:10:30.152356+0000","last_clean":"2024-08-16T19:10:30.152356+0000","last_became_active":"2024-08-16T19:08:02.457994+0000","last_became_peered":"2024-08-16T19:08:02.457994+0000","last_unstale":"2024-08-16T19:10:30.152356+0000","last_undegraded":"2024-08-16T19:10:30.152356+0000","last_fullsized":"2024-08-16T19:10:30.152356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:14:20.344959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395202+0000","last_change":"2024-08-16T19:08:11.623242+0000","last_active":"2024-08-16T19:10:28.395202+0000","last_peered":"2024-08-16T19:10:28.395202+0000","last_clean":"2024-08-16T19:10:28.395202+0000","last_became_active":"2024-08-16T19:08:11.622749+0000","last_became_peered":"2024-08-16T19:08:11.622749+0000","last_unstale":"2024-08-16T19:10:28.395202+0000","last_undegraded":"2024-08-16T19:10:28.395202+0000","last_fullsized":"2024-08-16T19:10:28.395202+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:35:28.446805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415903+0000","last_change":"2024-08-16T19:08:04.599286+0000","last_active":"2024-08-16T19:10:28.415903+0000","last_peered":"2024-08-16T19:10:28.415903+0000","last_clean":"2024-08-16T19:10:28.415903+0000","last_became_active":"2024-08-16T19:08:04.598922+0000","last_became_peered":"2024-08-16T19:08:04.598922+0000","last_unstale":"2024-08-16T19:10:28.415903+0000","last_undegraded":"2024-08-16T19:10:28.415903+0000","last_fullsized":"2024-08-16T19:10:28.415903+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:30:08.889469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309754+0000","last_change":"2024-08-16T19:09:59.590142+0000","last_active":"2024-08-16T19:10:28.309754+0000","last_peered":"2024-08-16T19:10:28.309754+0000","last_clean":"2024-08-16T19:10:28.309754+0000","last_became_active":"2024-08-16T19:09:59.580675+0000","last_became_peered":"2024-08-16T19:09:59.580675+0000","last_unstale":"2024-08-16T19:10:28.309754+0000","last_undegraded":"2024-08-16T19:10:28.309754+0000","last_fullsized":"2024-08-16T19:10:28.309754+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:06:11.306822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311414+0000","last_change":"2024-08-16T19:08:11.623969+0000","last_active":"2024-08-16T19:10:28.311414+0000","last_peered":"2024-08-16T19:10:28.311414+0000","last_clean":"2024-08-16T19:10:28.311414+0000","last_became_active":"2024-08-16T19:08:11.623605+0000","last_became_peered":"2024-08-16T19:08:11.623605+0000","last_unstale":"2024-08-16T19:10:28.311414+0000","last_undegraded":"2024-08-16T19:10:28.311414+0000","last_fullsized":"2024-08-16T19:10:28.311414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:28:19.117567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416700+0000","last_change":"2024-08-16T19:08:04.595284+0000","last_active":"2024-08-16T19:10:28.416700+0000","last_peered":"2024-08-16T19:10:28.416700+0000","last_clean":"2024-08-16T19:10:28.416700+0000","last_became_active":"2024-08-16T19:08:04.594909+0000","last_became_peered":"2024-08-16T19:08:04.594909+0000","last_unstale":"2024-08-16T19:10:28.416700+0000","last_undegraded":"2024-08-16T19:10:28.416700+0000","last_fullsized":"2024-08-16T19:10:28.416700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:19:03.996342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416400+0000","last_change":"2024-08-16T19:09:59.587138+0000","last_active":"2024-08-16T19:10:28.416400+0000","last_peered":"2024-08-16T19:10:28.416400+0000","last_clean":"2024-08-16T19:10:28.416400+0000","last_became_active":"2024-08-16T19:09:59.579143+0000","last_became_peered":"2024-08-16T19:09:59.579143+0000","last_unstale":"2024-08-16T19:10:28.416400+0000","last_undegraded":"2024-08-16T19:10:28.416400+0000","last_fullsized":"2024-08-16T19:10:28.416400+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:36:08.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311491+0000","last_change":"2024-08-16T19:08:11.622572+0000","last_active":"2024-08-16T19:10:28.311491+0000","last_peered":"2024-08-16T19:10:28.311491+0000","last_clean":"2024-08-16T19:10:28.311491+0000","last_became_active":"2024-08-16T19:08:11.621841+0000","last_became_peered":"2024-08-16T19:08:11.621841+0000","last_unstale":"2024-08-16T19:10:28.311491+0000","last_undegraded":"2024-08-16T19:10:28.311491+0000","last_fullsized":"2024-08-16T19:10:28.311491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:23.686600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311658+0000","last_change":"2024-08-16T19:08:04.598839+0000","last_active":"2024-08-16T19:10:28.311658+0000","last_peered":"2024-08-16T19:10:28.311658+0000","last_clean":"2024-08-16T19:10:28.311658+0000","last_became_active":"2024-08-16T19:08:04.598525+0000","last_became_peered":"2024-08-16T19:08:04.598525+0000","last_unstale":"2024-08-16T19:10:28.311658+0000","last_undegraded":"2024-08-16T19:10:28.311658+0000","last_fullsized":"2024-08-16T19:10:28.311658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:28:04.893005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404122+0000","last_change":"2024-08-16T19:08:02.977811+0000","last_active":"2024-08-16T19:10:28.404122+0000","last_peered":"2024-08-16T19:10:28.404122+0000","last_clean":"2024-08-16T19:10:28.404122+0000","last_became_active":"2024-08-16T19:08:02.977447+0000","last_became_peered":"2024-08-16T19:08:02.977447+0000","last_unstale":"2024-08-16T19:10:28.404122+0000","last_undegraded":"2024-08-16T19:10:28.404122+0000","last_fullsized":"2024-08-16T19:10:28.404122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:31.330263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309655+0000","last_change":"2024-08-16T19:08:11.631329+0000","last_active":"2024-08-16T19:10:28.309655+0000","last_peered":"2024-08-16T19:10:28.309655+0000","last_clean":"2024-08-16T19:10:28.309655+0000","last_became_active":"2024-08-16T19:08:11.630984+0000","last_became_peered":"2024-08-16T19:08:11.630984+0000","last_unstale":"2024-08-16T19:10:28.309655+0000","last_undegraded":"2024-08-16T19:10:28.309655+0000","last_fullsized":"2024-08-16T19:10:28.309655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:08:10.531641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415689+0000","last_change":"2024-08-16T19:08:04.605113+0000","last_active":"2024-08-16T19:10:28.415689+0000","last_peered":"2024-08-16T19:10:28.415689+0000","last_clean":"2024-08-16T19:10:28.415689+0000","last_became_active":"2024-08-16T19:08:04.604661+0000","last_became_peered":"2024-08-16T19:08:04.604661+0000","last_unstale":"2024-08-16T19:10:28.415689+0000","last_undegraded":"2024-08-16T19:10:28.415689+0000","last_fullsized":"2024-08-16T19:10:28.415689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:07:45.127656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:32.557492+0000","last_change":"2024-08-16T19:08:02.455413+0000","last_active":"2024-08-16T19:10:32.557492+0000","last_peered":"2024-08-16T19:10:32.557492+0000","last_clean":"2024-08-16T19:10:32.557492+0000","last_became_active":"2024-08-16T19:08:02.455053+0000","last_became_peered":"2024-08-16T19:08:02.455053+0000","last_unstale":"2024-08-16T19:10:32.557492+0000","last_undegraded":"2024-08-16T19:10:32.557492+0000","last_fullsized":"2024-08-16T19:10:32.557492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:30:50.440883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404506+0000","last_change":"2024-08-16T19:08:59.594588+0000","last_active":"2024-08-16T19:10:28.404506+0000","last_peered":"2024-08-16T19:10:28.404506+0000","last_clean":"2024-08-16T19:10:28.404506+0000","last_became_active":"2024-08-16T19:08:59.594219+0000","last_became_peered":"2024-08-16T19:08:59.594219+0000","last_unstale":"2024-08-16T19:10:28.404506+0000","last_undegraded":"2024-08-16T19:10:28.404506+0000","last_fullsized":"2024-08-16T19:10:28.404506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:50:27.045541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415670+0000","last_change":"2024-08-16T19:08:04.602028+0000","last_active":"2024-08-16T19:10:28.415670+0000","last_peered":"2024-08-16T19:10:28.415670+0000","last_clean":"2024-08-16T19:10:28.415670+0000","last_became_active":"2024-08-16T19:08:04.601698+0000","last_became_peered":"2024-08-16T19:08:04.601698+0000","last_unstale":"2024-08-16T19:10:28.415670+0000","last_undegraded":"2024-08-16T19:10:28.415670+0000","last_fullsized":"2024-08-16T19:10:28.415670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:10:18.736677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311811+0000","last_change":"2024-08-16T19:08:02.457016+0000","last_active":"2024-08-16T19:10:28.311811+0000","last_peered":"2024-08-16T19:10:28.311811+0000","last_clean":"2024-08-16T19:10:28.311811+0000","last_became_active":"2024-08-16T19:08:02.456643+0000","last_became_peered":"2024-08-16T19:08:02.456643+0000","last_unstale":"2024-08-16T19:10:28.311811+0000","last_undegraded":"2024-08-16T19:10:28.311811+0000","last_fullsized":"2024-08-16T19:10:28.311811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:17:08.966245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416055+0000","last_change":"2024-08-16T19:08:11.628731+0000","last_active":"2024-08-16T19:10:28.416055+0000","last_peered":"2024-08-16T19:10:28.416055+0000","last_clean":"2024-08-16T19:10:28.416055+0000","last_became_active":"2024-08-16T19:08:11.628340+0000","last_became_peered":"2024-08-16T19:08:11.628340+0000","last_unstale":"2024-08-16T19:10:28.416055+0000","last_undegraded":"2024-08-16T19:10:28.416055+0000","last_fullsized":"2024-08-16T19:10:28.416055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:18:58.257520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308976+0000","last_change":"2024-08-16T19:08:04.601854+0000","last_active":"2024-08-16T19:10:28.308976+0000","last_peered":"2024-08-16T19:10:28.308976+0000","last_clean":"2024-08-16T19:10:28.308976+0000","last_became_active":"2024-08-16T19:08:04.601554+0000","last_became_peered":"2024-08-16T19:08:04.601554+0000","last_unstale":"2024-08-16T19:10:28.308976+0000","last_undegraded":"2024-08-16T19:10:28.308976+0000","last_fullsized":"2024-08-16T19:10:28.308976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:25:35.285955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416259+0000","last_change":"2024-08-16T19:08:02.458854+0000","last_active":"2024-08-16T19:10:28.416259+0000","last_peered":"2024-08-16T19:10:28.416259+0000","last_clean":"2024-08-16T19:10:28.416259+0000","last_became_active":"2024-08-16T19:08:02.458006+0000","last_became_peered":"2024-08-16T19:08:02.458006+0000","last_unstale":"2024-08-16T19:10:28.416259+0000","last_undegraded":"2024-08-16T19:10:28.416259+0000","last_fullsized":"2024-08-16T19:10:28.416259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:54:51.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308667+0000","last_change":"2024-08-16T19:08:59.594545+0000","last_active":"2024-08-16T19:10:28.308667+0000","last_peered":"2024-08-16T19:10:28.308667+0000","last_clean":"2024-08-16T19:10:28.308667+0000","last_became_active":"2024-08-16T19:08:59.594012+0000","last_became_peered":"2024-08-16T19:08:59.594012+0000","last_unstale":"2024-08-16T19:10:28.308667+0000","last_undegraded":"2024-08-16T19:10:28.308667+0000","last_fullsized":"2024-08-16T19:10:28.308667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:04:16.279317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404420+0000","last_change":"2024-08-16T19:08:04.599176+0000","last_active":"2024-08-16T19:10:28.404420+0000","last_peered":"2024-08-16T19:10:28.404420+0000","last_clean":"2024-08-16T19:10:28.404420+0000","last_became_active":"2024-08-16T19:08:04.598762+0000","last_became_peered":"2024-08-16T19:08:04.598762+0000","last_unstale":"2024-08-16T19:10:28.404420+0000","last_undegraded":"2024-08-16T19:10:28.404420+0000","last_fullsized":"2024-08-16T19:10:28.404420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:34:17.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"93'41","reported_seq":298,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.564088+0000","last_change":"2024-08-16T19:08:02.460648+0000","last_active":"2024-08-16T19:10:28.564088+0000","last_peered":"2024-08-16T19:10:28.564088+0000","last_clean":"2024-08-16T19:10:28.564088+0000","last_became_active":"2024-08-16T19:08:02.460248+0000","last_became_peered":"2024-08-16T19:08:02.460248+0000","last_unstale":"2024-08-16T19:10:28.564088+0000","last_undegraded":"2024-08-16T19:10:28.564088+0000","last_fullsized":"2024-08-16T19:10:28.564088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:45:03.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68780,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":216,"num_read_kb":171,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400036+0000","last_change":"2024-08-16T19:08:11.628795+0000","last_active":"2024-08-16T19:10:28.400036+0000","last_peered":"2024-08-16T19:10:28.400036+0000","last_clean":"2024-08-16T19:10:28.400036+0000","last_became_active":"2024-08-16T19:08:11.628409+0000","last_became_peered":"2024-08-16T19:08:11.628409+0000","last_unstale":"2024-08-16T19:10:28.400036+0000","last_undegraded":"2024-08-16T19:10:28.400036+0000","last_fullsized":"2024-08-16T19:10:28.400036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:55:36.999338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415798+0000","last_change":"2024-08-16T19:08:04.603059+0000","last_active":"2024-08-16T19:10:28.415798+0000","last_peered":"2024-08-16T19:10:28.415798+0000","last_clean":"2024-08-16T19:10:28.415798+0000","last_became_active":"2024-08-16T19:08:04.601763+0000","last_became_peered":"2024-08-16T19:08:04.601763+0000","last_unstale":"2024-08-16T19:10:28.415798+0000","last_undegraded":"2024-08-16T19:10:28.415798+0000","last_fullsized":"2024-08-16T19:10:28.415798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:24:07.213342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312030+0000","last_change":"2024-08-16T19:08:02.976465+0000","last_active":"2024-08-16T19:10:28.312030+0000","last_peered":"2024-08-16T19:10:28.312030+0000","last_clean":"2024-08-16T19:10:28.312030+0000","last_became_active":"2024-08-16T19:08:02.976051+0000","last_became_peered":"2024-08-16T19:08:02.976051+0000","last_unstale":"2024-08-16T19:10:28.312030+0000","last_undegraded":"2024-08-16T19:10:28.312030+0000","last_fullsized":"2024-08-16T19:10:28.312030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:39:10.919363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528381+0000","last_change":"2024-08-16T19:08:11.627803+0000","last_active":"2024-08-16T19:10:28.528381+0000","last_peered":"2024-08-16T19:10:28.528381+0000","last_clean":"2024-08-16T19:10:28.528381+0000","last_became_active":"2024-08-16T19:08:11.627279+0000","last_became_peered":"2024-08-16T19:08:11.627279+0000","last_unstale":"2024-08-16T19:10:28.528381+0000","last_undegraded":"2024-08-16T19:10:28.528381+0000","last_fullsized":"2024-08-16T19:10:28.528381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:50:04.014607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395911+0000","last_change":"2024-08-16T19:08:59.460897+0000","last_active":"2024-08-16T19:10:28.395911+0000","last_peered":"2024-08-16T19:10:28.395911+0000","last_clean":"2024-08-16T19:10:28.395911+0000","last_became_active":"2024-08-16T19:08:59.460530+0000","last_became_peered":"2024-08-16T19:08:59.460530+0000","last_unstale":"2024-08-16T19:10:28.395911+0000","last_undegraded":"2024-08-16T19:10:28.395911+0000","last_fullsized":"2024-08-16T19:10:28.395911+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:32:15.220144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416395+0000","last_change":"2024-08-16T19:08:02.460313+0000","last_active":"2024-08-16T19:10:28.416395+0000","last_peered":"2024-08-16T19:10:28.416395+0000","last_clean":"2024-08-16T19:10:28.416395+0000","last_became_active":"2024-08-16T19:08:02.460002+0000","last_became_peered":"2024-08-16T19:08:02.460002+0000","last_unstale":"2024-08-16T19:10:28.416395+0000","last_undegraded":"2024-08-16T19:10:28.416395+0000","last_fullsized":"2024-08-16T19:10:28.416395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:30:56.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.399910+0000","last_change":"2024-08-16T19:08:11.625134+0000","last_active":"2024-08-16T19:10:28.399910+0000","last_peered":"2024-08-16T19:10:28.399910+0000","last_clean":"2024-08-16T19:10:28.399910+0000","last_became_active":"2024-08-16T19:08:11.624782+0000","last_became_peered":"2024-08-16T19:08:11.624782+0000","last_unstale":"2024-08-16T19:10:28.399910+0000","last_undegraded":"2024-08-16T19:10:28.399910+0000","last_fullsized":"2024-08-16T19:10:28.399910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:54:03.265246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416283+0000","last_change":"2024-08-16T19:08:04.597706+0000","last_active":"2024-08-16T19:10:28.416283+0000","last_peered":"2024-08-16T19:10:28.416283+0000","last_clean":"2024-08-16T19:10:28.416283+0000","last_became_active":"2024-08-16T19:08:04.597380+0000","last_became_peered":"2024-08-16T19:08:04.597380+0000","last_unstale":"2024-08-16T19:10:28.416283+0000","last_undegraded":"2024-08-16T19:10:28.416283+0000","last_fullsized":"2024-08-16T19:10:28.416283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:24:30.905303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":252,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.584792+0000","last_change":"2024-08-16T19:08:02.457707+0000","last_active":"2024-08-16T19:10:28.584792+0000","last_peered":"2024-08-16T19:10:28.584792+0000","last_clean":"2024-08-16T19:10:28.584792+0000","last_became_active":"2024-08-16T19:08:02.457289+0000","last_became_peered":"2024-08-16T19:08:02.457289+0000","last_unstale":"2024-08-16T19:10:28.584792+0000","last_undegraded":"2024-08-16T19:10:28.584792+0000","last_fullsized":"2024-08-16T19:10:28.584792+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:51.887709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404894+0000","last_change":"2024-08-16T19:08:11.629764+0000","last_active":"2024-08-16T19:10:28.404894+0000","last_peered":"2024-08-16T19:10:28.404894+0000","last_clean":"2024-08-16T19:10:28.404894+0000","last_became_active":"2024-08-16T19:08:11.629369+0000","last_became_peered":"2024-08-16T19:08:11.629369+0000","last_unstale":"2024-08-16T19:10:28.404894+0000","last_undegraded":"2024-08-16T19:10:28.404894+0000","last_fullsized":"2024-08-16T19:10:28.404894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:53:08.070509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395310+0000","last_change":"2024-08-16T19:08:11.627109+0000","last_active":"2024-08-16T19:10:28.395310+0000","last_peered":"2024-08-16T19:10:28.395310+0000","last_clean":"2024-08-16T19:10:28.395310+0000","last_became_active":"2024-08-16T19:08:11.626688+0000","last_became_peered":"2024-08-16T19:08:11.626688+0000","last_unstale":"2024-08-16T19:10:28.395310+0000","last_undegraded":"2024-08-16T19:10:28.395310+0000","last_fullsized":"2024-08-16T19:10:28.395310+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:12:05.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.400063+0000","last_change":"2024-08-16T19:08:04.589183+0000","last_active":"2024-08-16T19:10:28.400063+0000","last_peered":"2024-08-16T19:10:28.400063+0000","last_clean":"2024-08-16T19:10:28.400063+0000","last_became_active":"2024-08-16T19:08:04.588856+0000","last_became_peered":"2024-08-16T19:08:04.588856+0000","last_unstale":"2024-08-16T19:10:28.400063+0000","last_undegraded":"2024-08-16T19:10:28.400063+0000","last_fullsized":"2024-08-16T19:10:28.400063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:53:39.301710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.311640+0000","last_change":"2024-08-16T19:08:02.458933+0000","last_active":"2024-08-16T19:10:28.311640+0000","last_peered":"2024-08-16T19:10:28.311640+0000","last_clean":"2024-08-16T19:10:28.311640+0000","last_became_active":"2024-08-16T19:08:02.457886+0000","last_became_peered":"2024-08-16T19:08:02.457886+0000","last_unstale":"2024-08-16T19:10:28.311640+0000","last_undegraded":"2024-08-16T19:10:28.311640+0000","last_fullsized":"2024-08-16T19:10:28.311640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:21:22.257804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415754+0000","last_change":"2024-08-16T19:08:11.632947+0000","last_active":"2024-08-16T19:10:28.415754+0000","last_peered":"2024-08-16T19:10:28.415754+0000","last_clean":"2024-08-16T19:10:28.415754+0000","last_became_active":"2024-08-16T19:08:11.632568+0000","last_became_peered":"2024-08-16T19:08:11.632568+0000","last_unstale":"2024-08-16T19:10:28.415754+0000","last_undegraded":"2024-08-16T19:10:28.415754+0000","last_fullsized":"2024-08-16T19:10:28.415754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:08:33.658887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404650+0000","last_change":"2024-08-16T19:08:04.589224+0000","last_active":"2024-08-16T19:10:28.404650+0000","last_peered":"2024-08-16T19:10:28.404650+0000","last_clean":"2024-08-16T19:10:28.404650+0000","last_became_active":"2024-08-16T19:08:04.588656+0000","last_became_peered":"2024-08-16T19:08:04.588656+0000","last_unstale":"2024-08-16T19:10:28.404650+0000","last_undegraded":"2024-08-16T19:10:28.404650+0000","last_fullsized":"2024-08-16T19:10:28.404650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:37:57.747289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308849+0000","last_change":"2024-08-16T19:08:02.451575+0000","last_active":"2024-08-16T19:10:28.308849+0000","last_peered":"2024-08-16T19:10:28.308849+0000","last_clean":"2024-08-16T19:10:28.308849+0000","last_became_active":"2024-08-16T19:08:02.451233+0000","last_became_peered":"2024-08-16T19:08:02.451233+0000","last_unstale":"2024-08-16T19:10:28.308849+0000","last_undegraded":"2024-08-16T19:10:28.308849+0000","last_fullsized":"2024-08-16T19:10:28.308849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:36:50.427711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308584+0000","last_change":"2024-08-16T19:08:02.452613+0000","last_active":"2024-08-16T19:10:28.308584+0000","last_peered":"2024-08-16T19:10:28.308584+0000","last_clean":"2024-08-16T19:10:28.308584+0000","last_became_active":"2024-08-16T19:08:02.452263+0000","last_became_peered":"2024-08-16T19:08:02.452263+0000","last_unstale":"2024-08-16T19:10:28.308584+0000","last_undegraded":"2024-08-16T19:10:28.308584+0000","last_fullsized":"2024-08-16T19:10:28.308584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:20:45.858048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.312007+0000","last_change":"2024-08-16T19:08:04.595706+0000","last_active":"2024-08-16T19:10:28.312007+0000","last_peered":"2024-08-16T19:10:28.312007+0000","last_clean":"2024-08-16T19:10:28.312007+0000","last_became_active":"2024-08-16T19:08:04.595338+0000","last_became_peered":"2024-08-16T19:08:04.595338+0000","last_unstale":"2024-08-16T19:10:28.312007+0000","last_undegraded":"2024-08-16T19:10:28.312007+0000","last_fullsized":"2024-08-16T19:10:28.312007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:03:46.860114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416295+0000","last_change":"2024-08-16T19:08:11.629511+0000","last_active":"2024-08-16T19:10:28.416295+0000","last_peered":"2024-08-16T19:10:28.416295+0000","last_clean":"2024-08-16T19:10:28.416295+0000","last_became_active":"2024-08-16T19:08:11.629129+0000","last_became_peered":"2024-08-16T19:08:11.629129+0000","last_unstale":"2024-08-16T19:10:28.416295+0000","last_undegraded":"2024-08-16T19:10:28.416295+0000","last_fullsized":"2024-08-16T19:10:28.416295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:56:55.803279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:33.559204+0000","last_change":"2024-08-16T19:08:02.461333+0000","last_active":"2024-08-16T19:10:33.559204+0000","last_peered":"2024-08-16T19:10:33.559204+0000","last_clean":"2024-08-16T19:10:33.559204+0000","last_became_active":"2024-08-16T19:08:02.460972+0000","last_became_peered":"2024-08-16T19:08:02.460972+0000","last_unstale":"2024-08-16T19:10:33.559204+0000","last_undegraded":"2024-08-16T19:10:33.559204+0000","last_fullsized":"2024-08-16T19:10:33.559204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:01.434721+0000","last_clean_scrub_stamp":"2024-08-16T19:08:01.434721+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:13:37.697891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1402,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404156+0000","last_change":"2024-08-16T19:08:04.591994+0000","last_active":"2024-08-16T19:10:28.404156+0000","last_peered":"2024-08-16T19:10:28.404156+0000","last_clean":"2024-08-16T19:10:28.404156+0000","last_became_active":"2024-08-16T19:08:04.591539+0000","last_became_peered":"2024-08-16T19:08:04.591539+0000","last_unstale":"2024-08-16T19:10:28.404156+0000","last_undegraded":"2024-08-16T19:10:28.404156+0000","last_fullsized":"2024-08-16T19:10:28.404156+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:50:23.239900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416359+0000","last_change":"2024-08-16T19:08:11.625860+0000","last_active":"2024-08-16T19:10:28.416359+0000","last_peered":"2024-08-16T19:10:28.416359+0000","last_clean":"2024-08-16T19:10:28.416359+0000","last_became_active":"2024-08-16T19:08:11.624952+0000","last_became_peered":"2024-08-16T19:08:11.624952+0000","last_unstale":"2024-08-16T19:10:28.416359+0000","last_undegraded":"2024-08-16T19:10:28.416359+0000","last_fullsized":"2024-08-16T19:10:28.416359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:10.603727+0000","last_clean_scrub_stamp":"2024-08-16T19:08:10.603727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:31:02.126959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415784+0000","last_change":"2024-08-16T19:08:04.596971+0000","last_active":"2024-08-16T19:10:28.415784+0000","last_peered":"2024-08-16T19:10:28.415784+0000","last_clean":"2024-08-16T19:10:28.415784+0000","last_became_active":"2024-08-16T19:08:04.596637+0000","last_became_peered":"2024-08-16T19:08:04.596637+0000","last_unstale":"2024-08-16T19:10:28.415784+0000","last_undegraded":"2024-08-16T19:10:28.415784+0000","last_fullsized":"2024-08-16T19:10:28.415784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:48:55.697300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415826+0000","last_change":"2024-08-16T19:08:59.461546+0000","last_active":"2024-08-16T19:10:28.415826+0000","last_peered":"2024-08-16T19:10:28.415826+0000","last_clean":"2024-08-16T19:10:28.415826+0000","last_became_active":"2024-08-16T19:08:59.461094+0000","last_became_peered":"2024-08-16T19:08:59.461094+0000","last_unstale":"2024-08-16T19:10:28.415826+0000","last_undegraded":"2024-08-16T19:10:28.415826+0000","last_fullsized":"2024-08-16T19:10:28.415826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:40:22.541686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415629+0000","last_change":"2024-08-16T19:08:04.591755+0000","last_active":"2024-08-16T19:10:28.415629+0000","last_peered":"2024-08-16T19:10:28.415629+0000","last_clean":"2024-08-16T19:10:28.415629+0000","last_became_active":"2024-08-16T19:08:04.591447+0000","last_became_peered":"2024-08-16T19:08:04.591447+0000","last_unstale":"2024-08-16T19:10:28.415629+0000","last_undegraded":"2024-08-16T19:10:28.415629+0000","last_fullsized":"2024-08-16T19:10:28.415629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:26:26.695229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395262+0000","last_change":"2024-08-16T19:08:04.596219+0000","last_active":"2024-08-16T19:10:28.395262+0000","last_peered":"2024-08-16T19:10:28.395262+0000","last_clean":"2024-08-16T19:10:28.395262+0000","last_became_active":"2024-08-16T19:08:04.595895+0000","last_became_peered":"2024-08-16T19:08:04.595895+0000","last_unstale":"2024-08-16T19:10:28.395262+0000","last_undegraded":"2024-08-16T19:10:28.395262+0000","last_fullsized":"2024-08-16T19:10:28.395262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:35:43.629523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309213+0000","last_change":"2024-08-16T19:08:04.598326+0000","last_active":"2024-08-16T19:10:28.309213+0000","last_peered":"2024-08-16T19:10:28.309213+0000","last_clean":"2024-08-16T19:10:28.309213+0000","last_became_active":"2024-08-16T19:08:04.597282+0000","last_became_peered":"2024-08-16T19:08:04.597282+0000","last_unstale":"2024-08-16T19:10:28.309213+0000","last_undegraded":"2024-08-16T19:10:28.309213+0000","last_fullsized":"2024-08-16T19:10:28.309213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:21:41.694238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416309+0000","last_change":"2024-08-16T19:08:04.588654+0000","last_active":"2024-08-16T19:10:28.416309+0000","last_peered":"2024-08-16T19:10:28.416309+0000","last_clean":"2024-08-16T19:10:28.416309+0000","last_became_active":"2024-08-16T19:08:04.588274+0000","last_became_peered":"2024-08-16T19:08:04.588274+0000","last_unstale":"2024-08-16T19:10:28.416309+0000","last_undegraded":"2024-08-16T19:10:28.416309+0000","last_fullsized":"2024-08-16T19:10:28.416309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:24:44.514979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416854+0000","last_change":"2024-08-16T19:08:04.591583+0000","last_active":"2024-08-16T19:10:28.416854+0000","last_peered":"2024-08-16T19:10:28.416854+0000","last_clean":"2024-08-16T19:10:28.416854+0000","last_became_active":"2024-08-16T19:08:04.591250+0000","last_became_peered":"2024-08-16T19:08:04.591250+0000","last_unstale":"2024-08-16T19:10:28.416854+0000","last_undegraded":"2024-08-16T19:10:28.416854+0000","last_fullsized":"2024-08-16T19:10:28.416854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:49.146200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.406058+0000","last_change":"2024-08-16T19:08:04.596911+0000","last_active":"2024-08-16T19:10:28.406058+0000","last_peered":"2024-08-16T19:10:28.406058+0000","last_clean":"2024-08-16T19:10:28.406058+0000","last_became_active":"2024-08-16T19:08:04.595484+0000","last_became_peered":"2024-08-16T19:08:04.595484+0000","last_unstale":"2024-08-16T19:10:28.406058+0000","last_undegraded":"2024-08-16T19:10:28.406058+0000","last_fullsized":"2024-08-16T19:10:28.406058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:35:19.302060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415471+0000","last_change":"2024-08-16T19:08:59.594510+0000","last_active":"2024-08-16T19:10:28.415471+0000","last_peered":"2024-08-16T19:10:28.415471+0000","last_clean":"2024-08-16T19:10:28.415471+0000","last_became_active":"2024-08-16T19:08:59.594052+0000","last_became_peered":"2024-08-16T19:08:59.594052+0000","last_unstale":"2024-08-16T19:10:28.415471+0000","last_undegraded":"2024-08-16T19:10:28.415471+0000","last_fullsized":"2024-08-16T19:10:28.415471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:30.566631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.405067+0000","last_change":"2024-08-16T19:08:04.594351+0000","last_active":"2024-08-16T19:10:28.405067+0000","last_peered":"2024-08-16T19:10:28.405067+0000","last_clean":"2024-08-16T19:10:28.405067+0000","last_became_active":"2024-08-16T19:08:04.594030+0000","last_became_peered":"2024-08-16T19:08:04.594030+0000","last_unstale":"2024-08-16T19:10:28.405067+0000","last_undegraded":"2024-08-16T19:10:28.405067+0000","last_fullsized":"2024-08-16T19:10:28.405067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:01.108782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415591+0000","last_change":"2024-08-16T19:08:04.601038+0000","last_active":"2024-08-16T19:10:28.415591+0000","last_peered":"2024-08-16T19:10:28.415591+0000","last_clean":"2024-08-16T19:10:28.415591+0000","last_became_active":"2024-08-16T19:08:04.600619+0000","last_became_peered":"2024-08-16T19:08:04.600619+0000","last_unstale":"2024-08-16T19:10:28.415591+0000","last_undegraded":"2024-08-16T19:10:28.415591+0000","last_fullsized":"2024-08-16T19:10:28.415591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:36:54.673855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309143+0000","last_change":"2024-08-16T19:08:04.601641+0000","last_active":"2024-08-16T19:10:28.309143+0000","last_peered":"2024-08-16T19:10:28.309143+0000","last_clean":"2024-08-16T19:10:28.309143+0000","last_became_active":"2024-08-16T19:08:04.601281+0000","last_became_peered":"2024-08-16T19:08:04.601281+0000","last_unstale":"2024-08-16T19:10:28.309143+0000","last_undegraded":"2024-08-16T19:10:28.309143+0000","last_fullsized":"2024-08-16T19:10:28.309143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:20:47.063077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416085+0000","last_change":"2024-08-16T19:08:59.466874+0000","last_active":"2024-08-16T19:10:28.416085+0000","last_peered":"2024-08-16T19:10:28.416085+0000","last_clean":"2024-08-16T19:10:28.416085+0000","last_became_active":"2024-08-16T19:08:59.466377+0000","last_became_peered":"2024-08-16T19:08:59.466377+0000","last_unstale":"2024-08-16T19:10:28.416085+0000","last_undegraded":"2024-08-16T19:10:28.416085+0000","last_fullsized":"2024-08-16T19:10:28.416085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:42:40.045026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416507+0000","last_change":"2024-08-16T19:08:04.589321+0000","last_active":"2024-08-16T19:10:28.416507+0000","last_peered":"2024-08-16T19:10:28.416507+0000","last_clean":"2024-08-16T19:10:28.416507+0000","last_became_active":"2024-08-16T19:08:04.589000+0000","last_became_peered":"2024-08-16T19:08:04.589000+0000","last_unstale":"2024-08-16T19:10:28.416507+0000","last_undegraded":"2024-08-16T19:10:28.416507+0000","last_fullsized":"2024-08-16T19:10:28.416507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:34:21.326422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.401991+0000","last_change":"2024-08-16T19:08:04.591519+0000","last_active":"2024-08-16T19:10:28.401991+0000","last_peered":"2024-08-16T19:10:28.401991+0000","last_clean":"2024-08-16T19:10:28.401991+0000","last_became_active":"2024-08-16T19:08:04.591229+0000","last_became_peered":"2024-08-16T19:08:04.591229+0000","last_unstale":"2024-08-16T19:10:28.401991+0000","last_undegraded":"2024-08-16T19:10:28.401991+0000","last_fullsized":"2024-08-16T19:10:28.401991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:44:07.252814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416849+0000","last_change":"2024-08-16T19:08:04.593622+0000","last_active":"2024-08-16T19:10:28.416849+0000","last_peered":"2024-08-16T19:10:28.416849+0000","last_clean":"2024-08-16T19:10:28.416849+0000","last_became_active":"2024-08-16T19:08:04.593013+0000","last_became_peered":"2024-08-16T19:08:04.593013+0000","last_unstale":"2024-08-16T19:10:28.416849+0000","last_undegraded":"2024-08-16T19:10:28.416849+0000","last_fullsized":"2024-08-16T19:10:28.416849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T00:48:55.232857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.415320+0000","last_change":"2024-08-16T19:08:04.598593+0000","last_active":"2024-08-16T19:10:28.415320+0000","last_peered":"2024-08-16T19:10:28.415320+0000","last_clean":"2024-08-16T19:10:28.415320+0000","last_became_active":"2024-08-16T19:08:04.598297+0000","last_became_peered":"2024-08-16T19:08:04.598297+0000","last_unstale":"2024-08-16T19:10:28.415320+0000","last_undegraded":"2024-08-16T19:10:28.415320+0000","last_fullsized":"2024-08-16T19:10:28.415320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:52:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.394824+0000","last_change":"2024-08-16T19:08:04.599675+0000","last_active":"2024-08-16T19:10:28.394824+0000","last_peered":"2024-08-16T19:10:28.394824+0000","last_clean":"2024-08-16T19:10:28.394824+0000","last_became_active":"2024-08-16T19:08:04.599390+0000","last_became_peered":"2024-08-16T19:08:04.599390+0000","last_unstale":"2024-08-16T19:10:28.394824+0000","last_undegraded":"2024-08-16T19:10:28.394824+0000","last_fullsized":"2024-08-16T19:10:28.394824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:46:50.147962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395082+0000","last_change":"2024-08-16T19:08:04.598025+0000","last_active":"2024-08-16T19:10:28.395082+0000","last_peered":"2024-08-16T19:10:28.395082+0000","last_clean":"2024-08-16T19:10:28.395082+0000","last_became_active":"2024-08-16T19:08:04.597680+0000","last_became_peered":"2024-08-16T19:08:04.597680+0000","last_unstale":"2024-08-16T19:10:28.395082+0000","last_undegraded":"2024-08-16T19:10:28.395082+0000","last_fullsized":"2024-08-16T19:10:28.395082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:18:39.154971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309135+0000","last_change":"2024-08-16T19:08:04.589551+0000","last_active":"2024-08-16T19:10:28.309135+0000","last_peered":"2024-08-16T19:10:28.309135+0000","last_clean":"2024-08-16T19:10:28.309135+0000","last_became_active":"2024-08-16T19:08:04.589205+0000","last_became_peered":"2024-08-16T19:08:04.589205+0000","last_unstale":"2024-08-16T19:10:28.309135+0000","last_undegraded":"2024-08-16T19:10:28.309135+0000","last_fullsized":"2024-08-16T19:10:28.309135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T23:53:03.312976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528358+0000","last_change":"2024-08-16T19:08:04.598565+0000","last_active":"2024-08-16T19:10:28.528358+0000","last_peered":"2024-08-16T19:10:28.528358+0000","last_clean":"2024-08-16T19:10:28.528358+0000","last_became_active":"2024-08-16T19:08:04.598169+0000","last_became_peered":"2024-08-16T19:08:04.598169+0000","last_unstale":"2024-08-16T19:10:28.528358+0000","last_undegraded":"2024-08-16T19:10:28.528358+0000","last_fullsized":"2024-08-16T19:10:28.528358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:13:53.830439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416564+0000","last_change":"2024-08-16T19:08:04.605716+0000","last_active":"2024-08-16T19:10:28.416564+0000","last_peered":"2024-08-16T19:10:28.416564+0000","last_clean":"2024-08-16T19:10:28.416564+0000","last_became_active":"2024-08-16T19:08:04.604769+0000","last_became_peered":"2024-08-16T19:08:04.604769+0000","last_unstale":"2024-08-16T19:10:28.416564+0000","last_undegraded":"2024-08-16T19:10:28.416564+0000","last_fullsized":"2024-08-16T19:10:28.416564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:38:02.665294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.309002+0000","last_change":"2024-08-16T19:08:04.595329+0000","last_active":"2024-08-16T19:10:28.309002+0000","last_peered":"2024-08-16T19:10:28.309002+0000","last_clean":"2024-08-16T19:10:28.309002+0000","last_became_active":"2024-08-16T19:08:04.594761+0000","last_became_peered":"2024-08-16T19:08:04.594761+0000","last_unstale":"2024-08-16T19:10:28.309002+0000","last_undegraded":"2024-08-16T19:10:28.309002+0000","last_fullsized":"2024-08-16T19:10:28.309002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:53:38.119583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416294+0000","last_change":"2024-08-16T19:08:04.596831+0000","last_active":"2024-08-16T19:10:28.416294+0000","last_peered":"2024-08-16T19:10:28.416294+0000","last_clean":"2024-08-16T19:10:28.416294+0000","last_became_active":"2024-08-16T19:08:04.596517+0000","last_became_peered":"2024-08-16T19:08:04.596517+0000","last_unstale":"2024-08-16T19:10:28.416294+0000","last_undegraded":"2024-08-16T19:10:28.416294+0000","last_fullsized":"2024-08-16T19:10:28.416294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:41:12.273930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404467+0000","last_change":"2024-08-16T19:08:04.603446+0000","last_active":"2024-08-16T19:10:28.404467+0000","last_peered":"2024-08-16T19:10:28.404467+0000","last_clean":"2024-08-16T19:10:28.404467+0000","last_became_active":"2024-08-16T19:08:04.603094+0000","last_became_peered":"2024-08-16T19:08:04.603094+0000","last_unstale":"2024-08-16T19:10:28.404467+0000","last_undegraded":"2024-08-16T19:10:28.404467+0000","last_fullsized":"2024-08-16T19:10:28.404467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:06:39.301933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416036+0000","last_change":"2024-08-16T19:08:04.600834+0000","last_active":"2024-08-16T19:10:28.416036+0000","last_peered":"2024-08-16T19:10:28.416036+0000","last_clean":"2024-08-16T19:10:28.416036+0000","last_became_active":"2024-08-16T19:08:04.600502+0000","last_became_peered":"2024-08-16T19:08:04.600502+0000","last_unstale":"2024-08-16T19:10:28.416036+0000","last_undegraded":"2024-08-16T19:10:28.416036+0000","last_fullsized":"2024-08-16T19:10:28.416036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T01:15:44.048131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.308504+0000","last_change":"2024-08-16T19:08:04.594206+0000","last_active":"2024-08-16T19:10:28.308504+0000","last_peered":"2024-08-16T19:10:28.308504+0000","last_clean":"2024-08-16T19:10:28.308504+0000","last_became_active":"2024-08-16T19:08:04.593852+0000","last_became_peered":"2024-08-16T19:08:04.593852+0000","last_unstale":"2024-08-16T19:10:28.308504+0000","last_undegraded":"2024-08-16T19:10:28.308504+0000","last_fullsized":"2024-08-16T19:10:28.308504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T21:41:32.837826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.404114+0000","last_change":"2024-08-16T19:08:04.589427+0000","last_active":"2024-08-16T19:10:28.404114+0000","last_peered":"2024-08-16T19:10:28.404114+0000","last_clean":"2024-08-16T19:10:28.404114+0000","last_became_active":"2024-08-16T19:08:04.589108+0000","last_became_peered":"2024-08-16T19:08:04.589108+0000","last_unstale":"2024-08-16T19:10:28.404114+0000","last_undegraded":"2024-08-16T19:10:28.404114+0000","last_fullsized":"2024-08-16T19:10:28.404114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:03:47.855302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.528254+0000","last_change":"2024-08-16T19:08:04.594858+0000","last_active":"2024-08-16T19:10:28.528254+0000","last_peered":"2024-08-16T19:10:28.528254+0000","last_clean":"2024-08-16T19:10:28.528254+0000","last_became_active":"2024-08-16T19:08:04.594556+0000","last_became_peered":"2024-08-16T19:08:04.594556+0000","last_unstale":"2024-08-16T19:10:28.528254+0000","last_undegraded":"2024-08-16T19:10:28.528254+0000","last_fullsized":"2024-08-16T19:10:28.528254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T19:57:03.713891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395415+0000","last_change":"2024-08-16T19:08:04.590173+0000","last_active":"2024-08-16T19:10:28.395415+0000","last_peered":"2024-08-16T19:10:28.395415+0000","last_clean":"2024-08-16T19:10:28.395415+0000","last_became_active":"2024-08-16T19:08:04.589850+0000","last_became_peered":"2024-08-16T19:08:04.589850+0000","last_unstale":"2024-08-16T19:10:28.395415+0000","last_undegraded":"2024-08-16T19:10:28.395415+0000","last_fullsized":"2024-08-16T19:10:28.395415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:38:14.491609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.395414+0000","last_change":"2024-08-16T19:08:04.598244+0000","last_active":"2024-08-16T19:10:28.395414+0000","last_peered":"2024-08-16T19:10:28.395414+0000","last_clean":"2024-08-16T19:10:28.395414+0000","last_became_active":"2024-08-16T19:08:04.597862+0000","last_became_peered":"2024-08-16T19:08:04.597862+0000","last_unstale":"2024-08-16T19:10:28.395414+0000","last_undegraded":"2024-08-16T19:10:28.395414+0000","last_fullsized":"2024-08-16T19:10:28.395414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T20:48:45.112802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-08-16T19:10:28.416642+0000","last_change":"2024-08-16T19:08:04.593376+0000","last_active":"2024-08-16T19:10:28.416642+0000","last_peered":"2024-08-16T19:10:28.416642+0000","last_clean":"2024-08-16T19:10:28.416642+0000","last_became_active":"2024-08-16T19:08:04.592567+0000","last_became_peered":"2024-08-16T19:08:04.592567+0000","last_unstale":"2024-08-16T19:10:28.416642+0000","last_undegraded":"2024-08-16T19:10:28.416642+0000","last_fullsized":"2024-08-16T19:10:28.416642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-16T19:08:03.560676+0000","last_clean_scrub_stamp":"2024-08-16T19:08:03.560676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T22:27:05.655916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799803,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1434,"num_read_kb":930,"num_write":511,"num_write_kb":2390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2568192,"data_stored":2202777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":389,"ondisk_log_size":389,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":538317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776174,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28948,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716204,"statfs":{"total":95995035648,"available":95965392896,"internally_reserved":0,"allocated":1916928,"data_stored":1325101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1503232,"data_stored":934616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201335,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1490944,"data_stored":930876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1699840,"data_stored":1136020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":533729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29128,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716024,"statfs":{"total":95995035648,"available":95965208576,"internally_reserved":0,"allocated":2039808,"data_stored":1458382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":537413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1818624,"data_stored":1253964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1175552,"data_stored":604539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":603539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":535185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-16T19:10:35.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:10:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:35 smithi155 ceph-mon[37413]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 80 op/s 2024-08-16T19:10:36.050 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:36.435 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:36 smithi062 ceph-mon[32993]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 -- 172.21.15.62:0/1828985745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc103390 msgr2=0x7efecc10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 --2- 172.21.15.62:0/1828985745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc103390 0x7efecc10b880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efec0009900 tx=0x7efec002a480 comp rx=0 tx=0).stop 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 -- 172.21.15.62:0/1828985745 shutdown_connections 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 --2- 172.21.15.62:0/1828985745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 --2- 172.21.15.62:0/1828985745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc103390 0x7efecc10b880 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 --2- 172.21.15.62:0/1828985745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc102a70 0x7efecc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 -- 172.21.15.62:0/1828985745 >> 172.21.15.62:0/1828985745 conn(0x7efecc0fc4d0 msgr2=0x7efecc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 -- 172.21.15.62:0/1828985745 shutdown_connections 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.516+0000 7efed1149640 1 -- 172.21.15.62:0/1828985745 wait complete. 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.517+0000 7efed1149640 1 Processor -- start 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.517+0000 7efed1149640 1 -- start start 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 0x7efecc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efecc111ca0 con 0x7efecc102a70 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efecc111b20 con 0x7efecc103390 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efed1149640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efecc111e20 con 0x7efecc10bea0 2024-08-16T19:10:36.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44332/0 (socket says 172.21.15.62:44332) 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 -- 172.21.15.62:0/1876969257 learned_addr learned my addr 172.21.15.62:0/1876969257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efeca59c640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 0x7efecc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecb59e640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 -- 172.21.15.62:0/1876969257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 msgr2=0x7efecc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 -- 172.21.15.62:0/1876969257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 msgr2=0x7efecc1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 0x7efecc1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 -- 172.21.15.62:0/1876969257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc19d130 con 0x7efecc102a70 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efeca59c640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 0x7efecc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecb59e640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.518+0000 7efecad9d640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efebc00d950 tx=0x7efebc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.519+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc017070 con 0x7efecc102a70 2024-08-16T19:10:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.519+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc19d420 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.519+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efebc009c30 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.519+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc00f6b0 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.519+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc1a4d00 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.520+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efecc102e50 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.521+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7efebc0109d0 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.522+0000 7efeb3fff640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 0x7efeb40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.522+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7efebc0cf660 con 0x7efecc102a70 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.522+0000 7efeca59c640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 0x7efeb40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.523+0000 7efeca59c640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 0x7efeb40a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efecc10d3e0 tx=0x7efec0005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:36.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.525+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efebc009de0 con 0x7efecc102a70 2024-08-16T19:10:36.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:36 smithi161 ceph-mon[37576]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:36.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.740+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efecc1994e0 con 0x7efecc102a70 2024-08-16T19:10:36.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.740+0000 7efeb3fff640 1 -- 172.21.15.62:0/1876969257 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7efebc09bd80 con 0x7efecc102a70 2024-08-16T19:10:36.742 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:36.742 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-08-16T19:10:26:918579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:10:26.868424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:10:36.743 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-08-16T19:10:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.742+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 msgr2=0x7efeb40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.742+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 0x7efeb40a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efecc10d3e0 tx=0x7efec0005a20 comp rx=0 tx=0).stop 2024-08-16T19:10:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.742+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 msgr2=0x7efecc198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.742+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efebc00d950 tx=0x7efebc00de10 comp rx=0 tx=0).stop 2024-08-16T19:10:36.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 shutdown_connections 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efeb40a2d80 0x7efeb40a5240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efecc10bea0 0x7efecc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efecc103390 0x7efecc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 --2- 172.21.15.62:0/1876969257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efecc102a70 0x7efecc198180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 >> 172.21.15.62:0/1876969257 conn(0x7efecc0fc4d0 msgr2=0x7efecc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 shutdown_connections 2024-08-16T19:10:36.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:36.743+0000 7efed1149640 1 -- 172.21.15.62:0/1876969257 wait complete. 2024-08-16T19:10:36.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:36 smithi155 ceph-mon[37413]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-16T19:10:37.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:10:37.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:37 smithi062 ceph-mon[32993]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-08-16T19:10:37.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:37 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1876969257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:37.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:37 smithi161 ceph-mon[37576]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-08-16T19:10:37.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:37 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1876969257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:37 smithi155 ceph-mon[37413]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-08-16T19:10:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:37 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1876969257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.109+0000 7f84f620b640 1 -- 172.21.15.62:0/2443414499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 msgr2=0x7f84f0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.109+0000 7f84f620b640 1 --2- 172.21.15.62:0/2443414499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0101d10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009930 tx=0x7f84d802a420 comp rx=0 tx=0).stop 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 -- 172.21.15.62:0/2443414499 shutdown_connections 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 --2- 172.21.15.62:0/2443414499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f0102250 0x7f84f010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 --2- 172.21.15.62:0/2443414499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0101d10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 --2- 172.21.15.62:0/2443414499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f01078b0 0x7f84f0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 -- 172.21.15.62:0/2443414499 >> 172.21.15.62:0/2443414499 conn(0x7f84f00fd5f0 msgr2=0x7f84f00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 -- 172.21.15.62:0/2443414499 shutdown_connections 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 -- 172.21.15.62:0/2443414499 wait complete. 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.110+0000 7f84f620b640 1 Processor -- start 2024-08-16T19:10:38.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 -- start start 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0102250 0x7f84f0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 0x7f84f019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0111cb0 con 0x7f84f01018b0 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0111b30 con 0x7f84f0102250 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84f620b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0111e30 con 0x7f84f01078b0 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44364/0 (socket says 172.21.15.62:44364) 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 -- 172.21.15.62:0/3089865747 learned_addr learned my addr 172.21.15.62:0/3089865747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84e7fff640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0102250 0x7f84f0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84effff640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 0x7f84f019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 -- 172.21.15.62:0/3089865747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 msgr2=0x7f84f019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:38.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 0x7f84f019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 -- 172.21.15.62:0/3089865747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0102250 msgr2=0x7f84f0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0102250 0x7f84f0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 -- 172.21.15.62:0/3089865747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f019d1c0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84effff640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 0x7f84f019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.111+0000 7f84ef7fe640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c890 tx=0x7f84e000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.112+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004220 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.112+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e00043c0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.112+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f019d4b0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.112+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00049e0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.112+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01a4d90 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.113+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f0102ff0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.114+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f84e00079b0 con 0x7f84f01018b0 2024-08-16T19:10:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.114+0000 7f84ed7fa640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 0x7f84dc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:38.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.115+0000 7f84e7fff640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 0x7f84dc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:38.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.115+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f84e00cf980 con 0x7f84f01018b0 2024-08-16T19:10:38.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.115+0000 7f84e7fff640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 0x7f84dc0a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009900 tx=0x7f84d80058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:38.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.119+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f84e0014030 con 0x7f84f01018b0 2024-08-16T19:10:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.330+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84f0109e80 con 0x7f84f01018b0 2024-08-16T19:10:38.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.330+0000 7f84ed7fa640 1 -- 172.21.15.62:0/3089865747 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f84e009c0a0 con 0x7f84f01018b0 2024-08-16T19:10:38.332 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:38.332 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-08-16T19:10:26:918579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:10:26.868424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:10:38.333 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-08-16T19:10:38.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 msgr2=0x7f84dc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:38.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 0x7f84dc0a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009900 tx=0x7f84d80058d0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 msgr2=0x7f84f0198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c890 tx=0x7f84e000cd50 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 shutdown_connections 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f84dc0a2d80 0x7f84dc0a5240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f01078b0 0x7f84f019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84f0102250 0x7f84f0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 --2- 172.21.15.62:0/3089865747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84f01018b0 0x7f84f0198210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 >> 172.21.15.62:0/3089865747 conn(0x7f84f00fd5f0 msgr2=0x7f84f00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 shutdown_connections 2024-08-16T19:10:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:38.333+0000 7f84f620b640 1 -- 172.21.15.62:0/3089865747 wait complete. 2024-08-16T19:10:38.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:38 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3089865747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:38.650 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:38 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3089865747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:38.730 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:38 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3089865747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:39.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:10:39.292 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:39.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:39 smithi161 ceph-mon[37576]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-08-16T19:10:39.671 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:39 smithi062 ceph-mon[32993]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-08-16T19:10:39.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:39 smithi155 ceph-mon[37413]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-08-16T19:10:39.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 -- 172.21.15.62:0/1116161682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c103410 msgr2=0x7ff45c1078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 --2- 172.21.15.62:0/1116161682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c103410 0x7ff45c1078c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff450009930 tx=0x7ff45002a420 comp rx=0 tx=0).stop 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 -- 172.21.15.62:0/1116161682 shutdown_connections 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 --2- 172.21.15.62:0/1116161682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c107fd0 0x7ff45c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 --2- 172.21.15.62:0/1116161682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c103410 0x7ff45c1078c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 --2- 172.21.15.62:0/1116161682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c102a60 0x7ff45c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.751+0000 7ff46308a640 1 -- 172.21.15.62:0/1116161682 >> 172.21.15.62:0/1116161682 conn(0x7ff45c0fc500 msgr2=0x7ff45c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- 172.21.15.62:0/1116161682 shutdown_connections 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- 172.21.15.62:0/1116161682 wait complete. 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 Processor -- start 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- start start 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c102a60 0x7ff45c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c103410 0x7ff45c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c111e20 con 0x7ff45c107fd0 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c111ca0 con 0x7ff45c103410 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.752+0000 7ff46308a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c111fa0 con 0x7ff45c102a60 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44384/0 (socket says 172.21.15.62:44384) 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 -- 172.21.15.62:0/2932858802 learned_addr learned my addr 172.21.15.62:0/2932858802 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff460e26640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c102a60 0x7ff45c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff45bfff640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c103410 0x7ff45c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 -- 172.21.15.62:0/2932858802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c102a60 msgr2=0x7ff45c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c102a60 0x7ff45c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 -- 172.21.15.62:0/2932858802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c103410 msgr2=0x7ff45c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:39.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c103410 0x7ff45c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 -- 172.21.15.62:0/2932858802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c19d350 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff461627640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff45400cad0 tx=0x7ff45400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400ae20 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff454005020 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19d640 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.753+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400f6f0 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.754+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a4f20 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.754+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c102e40 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.756+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff4540051d0 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.756+0000 7ff459ffb640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 0x7ff4440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.756+0000 7ff460e26640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 0x7ff4440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.756+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7ff4540cfbb0 con 0x7ff45c107fd0 2024-08-16T19:10:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.757+0000 7ff460e26640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 0x7ff4440a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00bbe0 tx=0x7ff44c00b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:39.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.760+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff454014030 con 0x7ff45c107fd0 2024-08-16T19:10:39.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.936+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff45c199700 con 0x7ff45c107fd0 2024-08-16T19:10:39.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.939+0000 7ff459ffb640 1 -- 172.21.15.62:0/2932858802 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+25201 (secure 0 0 0) 0x7ff45c199700 con 0x7ff45c107fd0 2024-08-16T19:10:39.940 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:39.941 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":93,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:10:28.302167+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/151448061":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/795341867":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1715350407":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/439686452":"2024-08-17T19:10:28.302138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 msgr2=0x7ff4440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 0x7ff4440a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00bbe0 tx=0x7ff44c00b390 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 msgr2=0x7ff45c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff45400cad0 tx=0x7ff45400cf90 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 shutdown_connections 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff4440a2d80 0x7ff4440a5240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff45c107fd0 0x7ff45c19cc70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff45c103410 0x7ff45c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 --2- 172.21.15.62:0/2932858802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff45c102a60 0x7ff45c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 >> 172.21.15.62:0/2932858802 conn(0x7ff45c0fc500 msgr2=0x7ff45c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 shutdown_connections 2024-08-16T19:10:39.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:39.941+0000 7ff46308a640 1 -- 172.21.15.62:0/2932858802 wait complete. 2024-08-16T19:10:40.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:40 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2932858802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:40.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-16T19:10:40.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:40 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2932858802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:40.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:40 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2932858802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:10:40.847 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:41.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:41 smithi062 ceph-mon[32993]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-08-16T19:10:41.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:41 smithi161 ceph-mon[37576]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-08-16T19:10:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:41 smithi155 ceph-mon[37413]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-08-16T19:10:41.940 INFO:teuthology.orchestra.run.smithi062.stdout:{"run_dir":"/var/run/ceph"} 2024-08-16T19:10:41.940 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-16T19:10:41.941 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7387 2024-08-16T19:10:41.941 DEBUG:tasks.quiescer:Yielding 2024-08-16T19:10:41.941 INFO:teuthology.run_tasks:Running task workunit... 2024-08-16T19:10:41.956 INFO:tasks.workunit:Pulling workunits from ref 998c2f980771f4723d937d5cd7891f92dd116699 2024-08-16T19:10:41.956 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-16T19:10:41.956 INFO:tasks.workunit:timeout=3h 2024-08-16T19:10:41.956 INFO:tasks.workunit:cleanup=True 2024-08-16T19:10:41.957 DEBUG:teuthology.orchestra.run.smithi062:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:41.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:10:41.975 INFO:teuthology.orchestra.run.smithi062.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-08-16 19:08:55.100742669 +0000 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-08-16 19:08:55.100742669 +0000 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-08-16 19:10:26.110163115 +0000 2024-08-16T19:10:41.976 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-08-16 19:08:55.100742669 +0000 2024-08-16T19:10:41.976 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-16T19:10:41.976 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-16T19:10:42.050 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 998c2f980771f4723d937d5cd7891f92dd116699 2024-08-16T19:10:42.107 INFO:tasks.workunit.client.0.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-16T19:10:42.227 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 -- 172.21.15.62:0/1554543311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c102ac0 msgr2=0x7ff16c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 --2- 172.21.15.62:0/1554543311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c102ac0 0x7ff16c102ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009930 tx=0x7ff15c02a420 comp rx=0 tx=0).stop 2024-08-16T19:10:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 -- 172.21.15.62:0/1554543311 shutdown_connections 2024-08-16T19:10:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 --2- 172.21.15.62:0/1554543311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c107fc0 0x7ff16c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 --2- 172.21.15.62:0/1554543311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c1078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 --2- 172.21.15.62:0/1554543311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c102ac0 0x7ff16c102ea0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 -- 172.21.15.62:0/1554543311 >> 172.21.15.62:0/1554543311 conn(0x7ff16c0fc500 msgr2=0x7ff16c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 -- 172.21.15.62:0/1554543311 shutdown_connections 2024-08-16T19:10:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.723+0000 7ff17085d640 1 -- 172.21.15.62:0/1554543311 wait complete. 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 Processor -- start 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 -- start start 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 0x7ff16c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c111dc0 con 0x7ff16c107fc0 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c111c40 con 0x7ff16c103470 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.724+0000 7ff17085d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c111f40 con 0x7ff16c102ac0 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47396/0 (socket says 172.21.15.62:47396) 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 -- 172.21.15.62:0/953765085 learned_addr learned my addr 172.21.15.62:0/953765085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16ad9d640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 0x7ff16c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff169d9b640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 -- 172.21.15.62:0/953765085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 msgr2=0x7ff16c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 -- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 msgr2=0x7ff16c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 0x7ff16c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 -- 172.21.15.62:0/953765085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c19d2d0 con 0x7ff16c102ac0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff169d9b640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16ad9d640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 0x7ff16c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.725+0000 7ff16a59c640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009230 tx=0x7ff15c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.726+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c03f030 con 0x7ff16c102ac0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.726+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c003ad0 con 0x7ff16c102ac0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.726+0000 7ff17085d640 1 -- 172.21.15.62:0/953765085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c19d560 con 0x7ff16c102ac0 2024-08-16T19:10:42.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.726+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c035070 con 0x7ff16c102ac0 2024-08-16T19:10:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.726+0000 7ff17085d640 1 -- 172.21.15.62:0/953765085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1a4ea0 con 0x7ff16c102ac0 2024-08-16T19:10:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.727+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16c06a520 con 0x7ff16c102ac0 2024-08-16T19:10:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.730+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff15c042030 con 0x7ff16c102ac0 2024-08-16T19:10:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.730+0000 7ff15b7fe640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 0x7ff1540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.731+0000 7ff169d9b640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 0x7ff1540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.731+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7ff15c0f0050 con 0x7ff16c102ac0 2024-08-16T19:10:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.731+0000 7ff169d9b640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 0x7ff1540a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0feb10 tx=0x7ff160005d30 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.732+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff15c0bc710 con 0x7ff16c102ac0 2024-08-16T19:10:42.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.946+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff16c199680 con 0x7ff16c102ac0 2024-08-16T19:10:42.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.947+0000 7ff15b7fe640 1 -- 172.21.15.62:0/953765085 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7ff15c0bc010 con 0x7ff16c102ac0 2024-08-16T19:10:42.949 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:10:42.949 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-08-16T19:10:26:918579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:10:26.868424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:10:42.949 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 msgr2=0x7ff1540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 0x7ff1540a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0feb10 tx=0x7ff160005d30 comp rx=0 tx=0).stop 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 msgr2=0x7ff16c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009230 tx=0x7ff15c004270 comp rx=0 tx=0).stop 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 shutdown_connections 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff1540a2d80 0x7ff1540a5240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff16c107fc0 0x7ff16c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c103470 0x7ff16c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 --2- 172.21.15.62:0/953765085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff16c102ac0 0x7ff16c198320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 >> 172.21.15.62:0/953765085 conn(0x7ff16c0fc500 msgr2=0x7ff16c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 shutdown_connections 2024-08-16T19:10:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:42.949+0000 7ff1597fa640 1 -- 172.21.15.62:0/953765085 wait complete. 2024-08-16T19:10:43.563 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:43 smithi062 ceph-mon[32993]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-08-16T19:10:43.563 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:43 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/953765085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:43.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-16T19:08:05.356964+0000', 'modified': '2024-08-16T19:10:26.868424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24476, 'mds_1': 24469, 'mds_2': 24451, 'mds_3': 24407, 'mds_4': 24400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6839/659126409', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 659126409}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 659126409}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6841/3131957026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6840', 'nonce': 3131957026}, {'type': 'v1', 'addr': '172.21.15.62:6841', 'nonce': 3131957026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.161:6833/1822368312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 1822368312}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 1822368312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.155:6833/2391473540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 2391473540}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 2391473540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/3063588307', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 3063588307}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 3063588307}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3346321531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3346321531}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3346321531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/3410199866', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 3410199866}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 3410199866}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.161:6837/3769663917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 3769663917}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 3769663917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.161:6839/2650096635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 2650096635}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 2650096635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6839/2045382457', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 2045382457}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 2045382457}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24400, 24407, 24451, 24469, 24476]} 2024-08-16T19:10:43.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-16T19:10:43.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.l status 2024-08-16T19:10:43.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:43 smithi161 ceph-mon[37576]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-08-16T19:10:43.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:43 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/953765085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:43 smithi155 ceph-mon[37413]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-08-16T19:10:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:43 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/953765085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:10:43.899 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 -- 172.21.15.62:0/2852866044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0681022a0 msgr2=0x7fd0681026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 --2- 172.21.15.62:0/2852866044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0681022a0 0x7fd0681026e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009900 tx=0x7fd05c02a480 comp rx=0 tx=0).stop 2024-08-16T19:10:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 -- 172.21.15.62:0/2852866044 shutdown_connections 2024-08-16T19:10:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 --2- 172.21.15.62:0/2852866044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd068102c20 0x7fd06810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 --2- 172.21.15.62:0/2852866044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0681022a0 0x7fd0681026e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 --2- 172.21.15.62:0/2852866044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068100b50 0x7fd068100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 -- 172.21.15.62:0/2852866044 >> 172.21.15.62:0/2852866044 conn(0x7fd0680fc500 msgr2=0x7fd0680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 -- 172.21.15.62:0/2852866044 shutdown_connections 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.374+0000 7fd06cf12640 1 -- 172.21.15.62:0/2852866044 wait complete. 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.375+0000 7fd06cf12640 1 Processor -- start 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.375+0000 7fd06cf12640 1 -- start start 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0681022a0 0x7fd068197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068102c20 0x7fd06819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd068110cb0 con 0x7fd068100b50 2024-08-16T19:10:44.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd068110b30 con 0x7fd068102c20 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06cf12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd068110e30 con 0x7fd0681022a0 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42334/0 (socket says 172.21.15.62:42334) 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 -- 172.21.15.62:0/1669486712 learned_addr learned my addr 172.21.15.62:0/1669486712 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd065d9b640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0681022a0 0x7fd068197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd066d9d640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068102c20 0x7fd06819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 -- 172.21.15.62:0/1669486712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0681022a0 msgr2=0x7fd068197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0681022a0 0x7fd068197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 -- 172.21.15.62:0/1669486712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068102c20 msgr2=0x7fd06819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068102c20 0x7fd06819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 -- 172.21.15.62:0/1669486712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06819c0c0 con 0x7fd068100b50 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.376+0000 7fd06659c640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd058009970 tx=0x7fd058009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.377+0000 7fd0577fe640 1 -- 172.21.15.62:0/1669486712 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800d020 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.377+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06819c3b0 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.377+0000 7fd0577fe640 1 -- 172.21.15.62:0/1669486712 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058007d40 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.377+0000 7fd0577fe640 1 -- 172.21.15.62:0/1669486712 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800ccc0 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.377+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd068069ca0 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.379+0000 7fd0577fe640 1 -- 172.21.15.62:0/1669486712 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fd05800c460 con 0x7fd068100b50 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.379+0000 7fd06f310740 1 Processor -- start 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 -- start start 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8630 0x55625fde8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55625fd4d710 con 0x55625fdf3810 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55625fce5770 con 0x55625fde8630 2024-08-16T19:10:44.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06f310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55625fd496d0 con 0x55625fde8f50 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42340/0 (socket says 172.21.15.62:42340) 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 -- 172.21.15.62:0/1492638369 learned_addr learned my addr 172.21.15.62:0/1492638369 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd066d9d640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8630 0x55625fde8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06659c640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 -- 172.21.15.62:0/1492638369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 msgr2=0x55625fdf32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 -- 172.21.15.62:0/1492638369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8630 msgr2=0x55625fde8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8630 0x55625fde8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd065d9b640 1 -- 172.21.15.62:0/1492638369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625fcec3e0 con 0x55625fdf3810 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd06659c640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.380+0000 7fd0577fe640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 0x7fd0500a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd066d9d640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 0x7fd0500a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd065d9b640 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5915b2f75adbc405 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd0577fe640 1 -- 172.21.15.62:0/1669486712 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fd0580d0ce0 con 0x7fd068100b50 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd054ff9640 1 -- 172.21.15.62:0/1492638369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c035030 con 0x55625fdf3810 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd054ff9640 1 -- 172.21.15.62:0/1492638369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c040070 con 0x55625fdf3810 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.381+0000 7fd054ff9640 1 -- 172.21.15.62:0/1492638369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0319d0 con 0x55625fdf3810 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- 172.21.15.62:0/1492638369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 msgr2=0x55625fdf5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c0040c0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd066d9d640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 0x7fd0500a5240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd060005fd0 tx=0x7fd060009600 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- 172.21.15.62:0/1492638369 shutdown_connections 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fdf3810 0x55625fdf5c00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 --2- 172.21.15.62:0/1492638369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8630 0x55625fde8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- 172.21.15.62:0/1492638369 >> 172.21.15.62:0/1492638369 conn(0x55625fde4bc0 msgr2=0x55625fde4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- 172.21.15.62:0/1492638369 shutdown_connections 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- 172.21.15.62:0/1492638369 wait complete. 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 -- start start 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.382+0000 7fd06f310740 1 Processor -- start 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fdf1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55625fdf63e0 con 0x55625fde8630 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55625fd08970 con 0x55625fdf3810 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55625fdf6560 con 0x55625fde8f50 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06f310740 10 client.0 _ll_register_callbacks cb 0x55625fa5c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42352/0 (socket says 172.21.15.62:42352) 2024-08-16T19:10:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06659c640 1 -- 172.21.15.62:0/3970256441 learned_addr learned my addr 172.21.15.62:0/3970256441 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd066d9d640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fdf1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 -- 172.21.15.62:0/3970256441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 msgr2=0x55625fdf1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fdf1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 -- 172.21.15.62:0/3970256441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 msgr2=0x55625fdf0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 -- 172.21.15.62:0/3970256441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55625fabf2b0 con 0x55625fde8f50 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd06659c640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd065d9b640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf0bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006010 tx=0x7fd05c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d7bd51143d49ffd server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.383+0000 7fd04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c035050 con 0x55625fde8f50 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fd05c032c70 con 0x55625fde8f50 2024-08-16T19:10:44.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c040070 con 0x55625fde8f50 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd05c005020 con 0x55625fde8f50 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 10 client.25114 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55625fdf63e0 con 0x55625fde8f50 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05c040070 con 0x55625fde8f50 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd05c0035f0 con 0x55625fde8f50 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 10 client.25114 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 10 client.25114 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 20 client.25114 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe040b0 0x55625fe05520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 4 client.25114 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.384+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55625fd5ac80 con 0x55625fe040b0 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.385+0000 7fd06659c640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe040b0 0x55625fe05520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.385+0000 7fd04dffb640 10 client.25114 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.385+0000 7fd06659c640 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe040b0 0x55625fe05520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe04d1f4e4d8e684 server_cookie=3d755e0d530edabe in_seq=0 out_seq=0 2024-08-16T19:10:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd04dffb640 1 -- 172.21.15.62:0/3970256441 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55625fd5ac80 con 0x55625fe040b0 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd04dffb640 10 client.25114 handle_command_reply: tid=0 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 client.25114 shutdown 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 msgr2=0x55625fdf0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf0bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd05c006010 tx=0x7fd05c009590 comp rx=0 tx=0).stop 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 shutdown_connections 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe040b0 0x55625fe05520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fdf1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8f50 0x55625fdf0bc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 --2- 172.21.15.62:0/3970256441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdf0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 >> 172.21.15.62:0/3970256441 conn(0x55625fde4bc0 msgr2=0x55625fdea8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd04dffb640 0 client.25114 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 shutdown_connections 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 1 -- 172.21.15.62:0/3970256441 wait complete. 2024-08-16T19:10:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.386+0000 7fd06f310740 20 client.25114 trim_cache size 0 max 16384 2024-08-16T19:10:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 Processor -- start 2024-08-16T19:10:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 -- start start 2024-08-16T19:10:44.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 0x55625fdf0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 0x55625fdfeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55625fe03860 con 0x55625fde8630 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55625fe03f30 con 0x55625fde8f50 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06f310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55625fe04600 con 0x55625fdf3810 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42366/0 (socket says 172.21.15.62:42366) 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.404+0000 7fd06659c640 1 -- 172.21.15.62:0/2692661787 learned_addr learned my addr 172.21.15.62:0/2692661787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd065d9b640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 0x55625fdf0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd066d9d640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 0x55625fdfeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 -- 172.21.15.62:0/2692661787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 msgr2=0x55625fdfeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 0x55625fdfeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 -- 172.21.15.62:0/2692661787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 msgr2=0x55625fdf0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 0x55625fdf0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 -- 172.21.15.62:0/2692661787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625fde0130 con 0x55625fde8630 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd066d9d640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 0x55625fdfeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd065d9b640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 0x55625fdf0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd06659c640 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd0580ce6a0 tx=0x7fd058005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81105032a7faba2 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd04ffff640 1 -- 172.21.15.62:0/2692661787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0580a10b0 con 0x55625fde8630 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd04ffff640 1 -- 172.21.15.62:0/2692661787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0580a5970 con 0x55625fde8630 2024-08-16T19:10:44.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.405+0000 7fd04ffff640 1 -- 172.21.15.62:0/2692661787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05809c460 con 0x55625fde8630 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- 172.21.15.62:0/2692661787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 msgr2=0x55625fdeb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd0580ce6a0 tx=0x7fd058005880 comp rx=0 tx=0).stop 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- 172.21.15.62:0/2692661787 shutdown_connections 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fdf3810 0x55625fdfeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fde8f50 0x55625fdf0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- 172.21.15.62:0/2692661787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8630 0x55625fdeb090 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- 172.21.15.62:0/2692661787 >> 172.21.15.62:0/2692661787 conn(0x55625fde4bc0 msgr2=0x55625fdfe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- 172.21.15.62:0/2692661787 shutdown_connections 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- 172.21.15.62:0/2692661787 wait complete. 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- start start 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 Processor -- start 2024-08-16T19:10:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8630 0x55625fdec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fded5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55625fe053a0 con 0x55625fde8f50 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55625fe04cd0 con 0x55625fdf3810 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55625fe05a70 con 0x55625fde8630 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.406+0000 7fd06f310740 10 client.0 _ll_register_callbacks cb 0x7fd060009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42376/0 (socket says 172.21.15.62:42376) 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 -- 172.21.15.62:0/2326898083 learned_addr learned my addr 172.21.15.62:0/2326898083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd06659c640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8630 0x55625fdec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd066d9d640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fded5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 -- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8630 msgr2=0x55625fdec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8630 0x55625fdec8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 -- 172.21.15.62:0/2326898083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 msgr2=0x55625fded5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fded5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 -- 172.21.15.62:0/2326898083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55625fdeddf0 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd065d9b640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f99e1ad60f973e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c05f060 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fd05c032c70 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0580a5970 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd05c035040 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.407+0000 7fd06f310740 10 client.15447 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55625fe72f10 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0580a5970 con 0x55625fde8f50 2024-08-16T19:10:44.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd05c0047e0 con 0x55625fde8f50 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 10 client.15447 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 10 client.15447 resolve_mds: resolved name 'l' to daemon mds.l 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 20 client.15447 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe77890 0x55625fe765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 4 client.15447 mds_command: new command op to 24400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 --> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] -- command(tid 0: {"prefix": "status"}) -- 0x55625fd55190 con 0x55625fe77890 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.408+0000 7fd06659c640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe77890 0x55625fe765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd04dffb640 10 client.15447 ms_handle_connect on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06659c640 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe77890 0x55625fe765a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=471776d68e996d06 server_cookie=e7c2f9fbc44b31d in_seq=0 out_seq=0 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd04dffb640 1 -- 172.21.15.62:0/2326898083 <== mds.4 v2:172.21.15.161:6832/1822368312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55625fd55190 con 0x55625fe77890 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd04dffb640 10 client.15447 handle_command_reply: tid=0 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 client.15447 shutdown 2024-08-16T19:10:44.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 msgr2=0x55625fdece30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd05c02a9a0 tx=0x7fd05c0038a0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 shutdown_connections 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:6832/1822368312,v1:172.21.15.161:6833/1822368312] conn(0x55625fe77890 0x55625fe765a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55625fdf3810 0x55625fded5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55625fde8f50 0x55625fdece30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 --2- 172.21.15.62:0/2326898083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55625fde8630 0x55625fdec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 >> 172.21.15.62:0/2326898083 conn(0x55625fde4bc0 msgr2=0x55625fdfefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd04dffb640 0 client.15447 ms_handle_reset on v2:172.21.15.161:6832/1822368312 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 shutdown_connections 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 1 -- 172.21.15.62:0/2326898083 wait complete. 2024-08-16T19:10:44.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.409+0000 7fd06f310740 20 client.15447 trim_cache size 0 max 16384 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 4, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24400, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 116.99884969, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 28, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 93, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 92, 2024-08-16T19:10:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 201.238483097 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 msgr2=0x7fd0500a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 0x7fd0500a5240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd060005fd0 tx=0x7fd060009600 comp rx=0 tx=0).stop 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 msgr2=0x7fd068197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd058009970 tx=0x7fd058009e30 comp rx=0 tx=0).stop 2024-08-16T19:10:44.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 shutdown_connections 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd0500a2d80 0x7fd0500a5240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd068102c20 0x7fd06819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0681022a0 0x7fd068197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 --2- 172.21.15.62:0/1669486712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd068100b50 0x7fd068197110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 >> 172.21.15.62:0/1669486712 conn(0x7fd0680fc500 msgr2=0x7fd0681065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 shutdown_connections 2024-08-16T19:10:44.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:44.410+0000 7fd06cf12640 1 -- 172.21.15.62:0/1669486712 wait complete. 2024-08-16T19:10:45.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.k status 2024-08-16T19:10:45.391 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:45.616 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:45 smithi062 ceph-mon[32993]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-08-16T19:10:45.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:45 smithi161 ceph-mon[37576]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 -- 172.21.15.62:0/1813502974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 msgr2=0x7f200010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 --2- 172.21.15.62:0/1813502974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f200010d770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009ca0 tx=0x7f1ff802a950 comp rx=0 tx=0).stop 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 -- 172.21.15.62:0/1813502974 shutdown_connections 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 --2- 172.21.15.62:0/1813502974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f200010d770 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 --2- 172.21.15.62:0/1813502974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20001022a0 0x7f20001026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 --2- 172.21.15.62:0/1813502974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2000100b50 0x7f2000100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 -- 172.21.15.62:0/1813502974 >> 172.21.15.62:0/1813502974 conn(0x7f20000fc500 msgr2=0x7f20000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 -- 172.21.15.62:0/1813502974 shutdown_connections 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.809+0000 7f200601a640 1 -- 172.21.15.62:0/1813502974 wait complete. 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 Processor -- start 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 -- start start 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 0x7f2000196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20001022a0 0x7f2000197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110d50 con 0x7f2000102c20 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110bd0 con 0x7f20001022a0 2024-08-16T19:10:45.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f200601a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2000110ed0 con 0x7f2000100b50 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42386/0 (socket says 172.21.15.62:42386) 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.810+0000 7f1ffffff640 1 -- 172.21.15.62:0/1415755282 learned_addr learned my addr 172.21.15.62:0/1415755282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20001022a0 0x7f2000197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 0x7f2000196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 -- 172.21.15.62:0/1415755282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 msgr2=0x7f2000196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 0x7f2000196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 -- 172.21.15.62:0/1415755282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20001022a0 msgr2=0x7f2000197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20001022a0 0x7f2000197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 -- 172.21.15.62:0/1415755282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20001a67a0 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 0x7f2000196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffffff640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006010 tx=0x7f1ff802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffcff9640 1 -- 172.21.15.62:0/1415755282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8034040 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001a6a30 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffcff9640 1 -- 172.21.15.62:0/1415755282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff8003e00 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f1ffcff9640 1 -- 172.21.15.62:0/1415755282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8045480 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.811+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a6fe0 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f1ffcff9640 1 -- 172.21.15.62:0/1415755282 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f1ff8037020 con 0x7f2000102c20 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 Processor -- start 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 -- start start 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24f50 0x55befae2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55befad89710 con 0x55befae24f50 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55befad21770 con 0x55befae2f810 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f2008418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55befad856d0 con 0x55befae24630 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47476/0 (socket says 172.21.15.62:47476) 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.814+0000 7f1ffeffd640 1 -- 172.21.15.62:0/3235323286 learned_addr learned my addr 172.21.15.62:0/3235323286 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24f50 0x55befae2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 -- 172.21.15.62:0/3235323286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 msgr2=0x55befae31c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae31c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 -- 172.21.15.62:0/3235323286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24f50 msgr2=0x55befae2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24f50 0x55befae2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 -- 172.21.15.62:0/3235323286 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befad283e0 con 0x55befae24630 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006e10 tx=0x7f1ff4009420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=608e95795fb5cb43 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffcff9640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 0x7f1fe80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1fe5ffb640 1 -- 172.21.15.62:0/3235323286 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff401c070 con 0x55befae24630 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1fe5ffb640 1 -- 172.21.15.62:0/3235323286 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4010040 con 0x55befae24630 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1fe5ffb640 1 -- 172.21.15.62:0/3235323286 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40146f0 con 0x55befae24630 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 0x7f1fe80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.815+0000 7f1ffcff9640 1 -- 172.21.15.62:0/1415755282 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1ff80f0a90 con 0x7f2000102c20 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 0x7f1fe80a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55befae21a20 tx=0x7f1ff0008040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- 172.21.15.62:0/3235323286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 msgr2=0x55befae24a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4006e10 tx=0x7f1ff4009420 comp rx=0 tx=0).stop 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- 172.21.15.62:0/3235323286 shutdown_connections 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae31c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24f50 0x55befae2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 --2- 172.21.15.62:0/3235323286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae24a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- 172.21.15.62:0/3235323286 >> 172.21.15.62:0/3235323286 conn(0x55befae20bc0 msgr2=0x55befae20fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- 172.21.15.62:0/3235323286 shutdown_connections 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- 172.21.15.62:0/3235323286 wait complete. 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.816+0000 7f2008418740 1 -- start start 2024-08-16T19:10:45.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 Processor -- start 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 0x55befae2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae2d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55befae323e0 con 0x55befae24630 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55befad44970 con 0x55befae2f810 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55befae32560 con 0x55befae24f50 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f2008418740 10 client.0 _ll_register_callbacks cb 0x55befaa987b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 0x55befae2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 0x55befae2cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47488/0 (socket says 172.21.15.62:47488) 2024-08-16T19:10:45.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42402/0 (socket says 172.21.15.62:42402) 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1ffeffd640 1 -- 172.21.15.62:0/2887612009 learned_addr learned my addr 172.21.15.62:0/2887612009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1ffffff640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 -- 172.21.15.62:0/2887612009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 msgr2=0x55befae2cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 0x55befae2cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 -- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 msgr2=0x55befae2d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae2d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1fff7fe640 1 -- 172.21.15.62:0/2887612009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befaafb2b0 con 0x55befae24630 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.817+0000 7f1ffffff640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae2d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e5a0 tx=0x7f1ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488fd1a011d8c947 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001a050 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1ff0014790 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff4010040 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ff00059c0 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f2008418740 10 client.15459 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55befae3e080 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.818+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4010040 con 0x55befae24630 2024-08-16T19:10:45.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1ff00063b0 con 0x55befae24630 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 10 client.15459 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 10 client.15459 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 20 client.15459 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befae40730 0x55befae41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 4 client.15459 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55befad96c80 con 0x55befae40730 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.819+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befae40730 0x55befae41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.820+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befae40730 0x55befae41ba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72b94fe47d74c379 server_cookie=ef6913f0f997de60 in_seq=0 out_seq=0 2024-08-16T19:10:45.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.820+0000 7f1fdeffd640 10 client.15459 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:45.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:45 smithi155 ceph-mon[37413]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 52 op/s 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.820+0000 7f1fdeffd640 1 -- 172.21.15.62:0/2887612009 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55befad96c80 con 0x55befae40730 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.820+0000 7f1fdeffd640 10 client.15459 handle_command_reply: tid=0 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.820+0000 7f2008418740 1 client.15459 shutdown 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 msgr2=0x55befae2c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000e5a0 tx=0x7f1ff00046c0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 shutdown_connections 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae2f810 0x55befae2d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befae40730 0x55befae41ba0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24f50 0x55befae2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 --2- 172.21.15.62:0/2887612009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae2c700 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 >> 172.21.15.62:0/2887612009 conn(0x55befae20bc0 msgr2=0x55befae268f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f1fdeffd640 0 client.15459 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 shutdown_connections 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 1 -- 172.21.15.62:0/2887612009 wait complete. 2024-08-16T19:10:45.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.821+0000 7f2008418740 20 client.15459 trim_cache size 0 max 16384 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 Processor -- start 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 -- start start 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55befae3fee0 con 0x55befae2f810 2024-08-16T19:10:45.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55befae405b0 con 0x55befae24f50 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f2008418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55befae40c80 con 0x55befae24630 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42408/0 (socket says 172.21.15.62:42408) 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 -- 172.21.15.62:0/881400184 learned_addr learned my addr 172.21.15.62:0/881400184 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 -- 172.21.15.62:0/881400184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 msgr2=0x55befae414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 -- 172.21.15.62:0/881400184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 msgr2=0x55befae270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.839+0000 7f1ffffff640 1 -- 172.21.15.62:0/881400184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befae2b6b0 con 0x55befae2f810 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f1ffffff640 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80ee450 tx=0x7f1ff8004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d258a1f0a8804945 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f1fe57fa640 1 -- 172.21.15.62:0/881400184 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff802c040 con 0x55befae2f810 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f1fe57fa640 1 -- 172.21.15.62:0/881400184 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff80c0f70 con 0x55befae2f810 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f1fe57fa640 1 -- 172.21.15.62:0/881400184 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80bce20 con 0x55befae2f810 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 -- 172.21.15.62:0/881400184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 msgr2=0x55befae3aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80ee450 tx=0x7f1ff8004480 comp rx=0 tx=0).stop 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 -- 172.21.15.62:0/881400184 shutdown_connections 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae2f810 0x55befae3aef0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 --2- 172.21.15.62:0/881400184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae24630 0x55befae414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.840+0000 7f2008418740 1 -- 172.21.15.62:0/881400184 >> 172.21.15.62:0/881400184 conn(0x55befae20bc0 msgr2=0x55befae3a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- 172.21.15.62:0/881400184 shutdown_connections 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- 172.21.15.62:0/881400184 wait complete. 2024-08-16T19:10:45.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- start start 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 Processor -- start 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae43750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 0x55befae43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55befae41a20 con 0x55befae24630 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55befae41350 con 0x55befae24f50 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55befae420f0 con 0x55befae2f810 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.841+0000 7f2008418740 10 client.0 _ll_register_callbacks cb 0x7f1ff000a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42412/0 (socket says 172.21.15.62:42412) 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 -- 172.21.15.62:0/3227390748 learned_addr learned my addr 172.21.15.62:0/3227390748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae43750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1ffffff640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 0x55befae43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 -- 172.21.15.62:0/3227390748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 msgr2=0x55befae43c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 0x55befae43c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 -- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 msgr2=0x55befae43750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae43750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 -- 172.21.15.62:0/3227390748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befae44350 con 0x55befae24630 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1ffffff640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 0x55befae43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fff7fe640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000a3e0 tx=0x7f1ff0009080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9c85a213a43a3b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001a040 con 0x55befae24630 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1ff001f910 con 0x55befae24630 2024-08-16T19:10:45.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.842+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff80c0f70 con 0x55befae24630 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ff001fb10 con 0x55befae24630 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 10 client.15465 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55befaeaae40 con 0x55befae24630 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff80c0f70 con 0x55befae24630 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1ff0032460 con 0x55befae24630 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 10 client.15465 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 10 client.15465 resolve_mds: resolved name 'k' to daemon mds.k 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 20 client.15465 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befaeaf930 0x55befaeae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 4 client.15465 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:45.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 --> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] -- command(tid 0: {"prefix": "status"}) -- 0x55befad91190 con 0x55befaeaf930 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.843+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befaeaf930 0x55befaeae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f1fdeffd640 10 client.15465 ms_handle_connect on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f1ffeffd640 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befaeaf930 0x55befaeae640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a651ca576e31fb96 server_cookie=f397646ae22993ee in_seq=0 out_seq=0 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f1fdeffd640 1 -- 172.21.15.62:0/3227390748 <== mds.3 v2:172.21.15.155:6832/2391473540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55befad91190 con 0x55befaeaf930 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f1fdeffd640 10 client.15465 handle_command_reply: tid=0 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 client.15465 shutdown 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 msgr2=0x55befae4a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000a3e0 tx=0x7f1ff0009080 comp rx=0 tx=0).stop 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 shutdown_connections 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:6832/2391473540,v1:172.21.15.155:6833/2391473540] conn(0x55befaeaf930 0x55befaeae640 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55befae2f810 0x55befae43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55befae24f50 0x55befae43750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f1fdeffd640 0 client.15465 ms_handle_reset on v2:172.21.15.155:6832/2391473540 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 --2- 172.21.15.62:0/3227390748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55befae24630 0x55befae4a420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 >> 172.21.15.62:0/3227390748 conn(0x55befae20bc0 msgr2=0x55befae3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 shutdown_connections 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.844+0000 7f2008418740 1 -- 172.21.15.62:0/3227390748 wait complete. 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f2008418740 20 client.15465 trim_cache size 0 max 16384 2024-08-16T19:10:45.854 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 3, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24407, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 119.440042956, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 28, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 93, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 92, 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 200.012883612 2024-08-16T19:10:45.855 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 msgr2=0x7f1fe80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 0x7f1fe80a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55befae21a20 tx=0x7f1ff0008040 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 msgr2=0x7f20001979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8006010 tx=0x7f1ff802f6f0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 shutdown_connections 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1fe80a2d80 0x7f1fe80a5240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2000102c20 0x7f20001979b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20001022a0 0x7f2000197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 --2- 172.21.15.62:0/1415755282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2000100b50 0x7f2000196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 >> 172.21.15.62:0/1415755282 conn(0x7f20000fc500 msgr2=0x7f20000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 shutdown_connections 2024-08-16T19:10:45.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:45.845+0000 7f200601a640 1 -- 172.21.15.62:0/1415755282 wait complete. 2024-08-16T19:10:46.563 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-16T19:10:46.827 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 -- 172.21.15.62:0/1948992912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4102a70 msgr2=0x7f55b4102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1948992912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4102a70 0x7f55b4102e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009900 tx=0x7f55a402a480 comp rx=0 tx=0).stop 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 -- 172.21.15.62:0/1948992912 shutdown_connections 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1948992912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b410bea0 0x7f55b410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1948992912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4103390 0x7f55b410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1948992912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b4102a70 0x7f55b4102e50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 -- 172.21.15.62:0/1948992912 >> 172.21.15.62:0/1948992912 conn(0x7f55b40fc4d0 msgr2=0x7f55b40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 -- 172.21.15.62:0/1948992912 shutdown_connections 2024-08-16T19:10:47.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.286+0000 7f55b8d15640 1 -- 172.21.15.62:0/1948992912 wait complete. 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 Processor -- start 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 -- start start 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 0x7f55b41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b4103390 0x7f55b41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111ce0 con 0x7f55b410bea0 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111b60 con 0x7f55b4102a70 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b8d15640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55b4111e60 con 0x7f55b4103390 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42440/0 (socket says 172.21.15.62:42440) 2024-08-16T19:10:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.287+0000 7f55b2d9d640 1 -- 172.21.15.62:0/1013592357 learned_addr learned my addr 172.21.15.62:0/1013592357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b4103390 0x7f55b41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b259c640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 0x7f55b41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 -- 172.21.15.62:0/1013592357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b4103390 msgr2=0x7f55b41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b4103390 0x7f55b41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 -- 172.21.15.62:0/1013592357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 msgr2=0x7f55b41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 0x7f55b41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 -- 172.21.15.62:0/1013592357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b419d150 con 0x7f55b410bea0 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b259c640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 0x7f55b41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b2d9d640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00cce0 tx=0x7f55ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55a37fe640 1 -- 172.21.15.62:0/1013592357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00f450 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55a37fe640 1 -- 172.21.15.62:0/1013592357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ac00fad0 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b419d440 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55a37fe640 1 -- 172.21.15.62:0/1013592357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac018c10 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.288+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b41a4d20 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55a37fe640 1 -- 172.21.15.62:0/1013592357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f55ac00f5f0 con 0x7f55b410bea0 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55bb113740 1 Processor -- start 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55bb113740 1 -- start start 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55bb113740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55bb113740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.291+0000 7f55bb113740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910ed010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55bb113740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c391040710 con 0x55c3910db510 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55bb113740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c390fd8770 con 0x55c3910dbe30 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55bb113740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c39103c6d0 con 0x55c3910eac20 2024-08-16T19:10:47.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55a37fe640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 0x7f559c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42450/0 (socket says 172.21.15.62:42450) 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 -- 172.21.15.62:0/4023433169 learned_addr learned my addr 172.21.15.62:0/4023433169 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b259c640 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b2d9d640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 0x7f559c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55a37fe640 1 -- 172.21.15.62:0/1013592357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55ac0d0070 con 0x7f55b410bea0 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 -- 172.21.15.62:0/4023433169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 msgr2=0x55c3910ed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910ed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 -- 172.21.15.62:0/4023433169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 msgr2=0x55c3910ea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910ea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 -- 172.21.15.62:0/4023433169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c390fdf3e0 con 0x55c3910db510 2024-08-16T19:10:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006aa0 tx=0x7f55a80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff2911b7c61c61e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55b2d9d640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 0x7f559c0a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0cd3d0 tx=0x7f55ac06a040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.292+0000 7f55a0ff9640 1 -- 172.21.15.62:0/4023433169 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a80127a0 con 0x55c3910db510 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55a0ff9640 1 -- 172.21.15.62:0/4023433169 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a801d070 con 0x55c3910db510 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55a0ff9640 1 -- 172.21.15.62:0/4023433169 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8018a70 con 0x55c3910db510 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 -- 172.21.15.62:0/4023433169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 msgr2=0x55c3910db8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006aa0 tx=0x7f55a80095c0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 -- 172.21.15.62:0/4023433169 shutdown_connections 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 --2- 172.21.15.62:0/4023433169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910db8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 -- 172.21.15.62:0/4023433169 >> 172.21.15.62:0/4023433169 conn(0x55c3910d7bc0 msgr2=0x55c3910d7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 -- 172.21.15.62:0/4023433169 shutdown_connections 2024-08-16T19:10:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.293+0000 7f55bb113740 1 -- 172.21.15.62:0/4023433169 wait complete. 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 -- start start 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 Processor -- start 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910db510 0x55c3910e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c3910ed7f0 con 0x55c3910dbe30 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c390ffb970 con 0x55c3910db510 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3910ed970 con 0x55c3910eac20 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55bb113740 10 client.0 _ll_register_callbacks cb 0x55c390d4f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42460/0 (socket says 172.21.15.62:42460) 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 -- 172.21.15.62:0/1610037621 learned_addr learned my addr 172.21.15.62:0/1610037621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910db510 0x55c3910e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 -- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 msgr2=0x55c3910e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 -- 172.21.15.62:0/1610037621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910db510 msgr2=0x55c3910e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910db510 0x55c3910e7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b259c640 1 -- 172.21.15.62:0/1610037621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c390db22b0 con 0x55c3910dbe30 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.294+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f55b259c640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55a402a9a0 tx=0x7f55a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c804d0704810ee08 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f5599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4035040 con 0x55c3910dbe30 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a801d070 con 0x55c3910dbe30 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55a4005020 con 0x55c3910dbe30 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55a404e050 con 0x55c3910dbe30 2024-08-16T19:10:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f55bb113740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.295+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3910f93c0 con 0x55c3910dbe30 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a801d070 con 0x55c3910dbe30 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f55a40035f0 con 0x55c3910dbe30 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 10 client.15477 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 20 client.15477 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c3910fb9c0 0x55c3910fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 4 client.15477 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:47.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.296+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39104dc80 con 0x55c3910fb9c0 2024-08-16T19:10:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.297+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c3910fb9c0 0x55c3910fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.297+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c3910fb9c0 0x55c3910fce30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1afeab301f3b420 server_cookie=3ef00bac075d6939 in_seq=0 out_seq=0 2024-08-16T19:10:47.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.297+0000 7f5599ffb640 10 client.15477 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f5599ffb640 1 -- 172.21.15.62:0/1610037621 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c39104dc80 con 0x55c3910fb9c0 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f5599ffb640 10 client.15477 handle_command_reply: tid=0 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 client.15477 shutdown 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 msgr2=0x55c3910e7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f55a402a9a0 tx=0x7f55a4004270 comp rx=0 tx=0).stop 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 shutdown_connections 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910eac20 0x55c3910e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910dbe30 0x55c3910e7fd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c3910fb9c0 0x55c3910fce30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 --2- 172.21.15.62:0/1610037621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910db510 0x55c3910e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f5599ffb640 0 client.15477 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.298+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 >> 172.21.15.62:0/1610037621 conn(0x55c3910d7bc0 msgr2=0x55c3910dd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.299+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 shutdown_connections 2024-08-16T19:10:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.299+0000 7f55bb113740 1 -- 172.21.15.62:0/1610037621 wait complete. 2024-08-16T19:10:47.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.299+0000 7f55bb113740 20 client.15477 trim_cache size 0 max 16384 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.316+0000 7f55bb113740 1 Processor -- start 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.316+0000 7f55bb113740 1 -- start start 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910db510 0x55c3910fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fb170 con 0x55c3910eac20 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fb840 con 0x55c3910dbe30 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55bb113740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fbf10 con 0x55c3910db510 2024-08-16T19:10:47.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42466/0 (socket says 172.21.15.62:42466) 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 -- 172.21.15.62:0/3829726317 learned_addr learned my addr 172.21.15.62:0/3829726317 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910db510 0x55c3910fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b259c640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 -- 172.21.15.62:0/3829726317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910db510 msgr2=0x55c3910fc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910db510 0x55c3910fc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 -- 172.21.15.62:0/3829726317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 msgr2=0x55c3910e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 -- 172.21.15.62:0/3829726317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3910e28b0 con 0x55c3910eac20 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b259c640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55c3910fd980 tx=0x7f55a8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81e424354afef2 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.317+0000 7f559bfff640 1 -- 172.21.15.62:0/3829726317 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8019b60 con 0x55c3910eac20 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f559bfff640 1 -- 172.21.15.62:0/3829726317 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8032070 con 0x55c3910eac20 2024-08-16T19:10:47.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f559bfff640 1 -- 172.21.15.62:0/3829726317 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8018a40 con 0x55c3910eac20 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- 172.21.15.62:0/3829726317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 msgr2=0x55c3910e6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55c3910fd980 tx=0x7f55a8009d20 comp rx=0 tx=0).stop 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- 172.21.15.62:0/3829726317 shutdown_connections 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910eac20 0x55c3910e6a70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910dbe30 0x55c3910e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 --2- 172.21.15.62:0/3829726317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910db510 0x55c3910fc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- 172.21.15.62:0/3829726317 >> 172.21.15.62:0/3829726317 conn(0x55c3910d7bc0 msgr2=0x55c3910e6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- 172.21.15.62:0/3829726317 shutdown_connections 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- 172.21.15.62:0/3829726317 wait complete. 2024-08-16T19:10:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.318+0000 7f55bb113740 1 -- start start 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 Processor -- start 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 0x55c3910fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910eac20 0x55c3911640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fccb0 con 0x55c3910db510 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fc5e0 con 0x55c3910eac20 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c3910fd380 con 0x55c3910dbe30 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55bb113740 10 client.0 _ll_register_callbacks cb 0x7f55a4031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42468/0 (socket says 172.21.15.62:42468) 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 -- 172.21.15.62:0/829671082 learned_addr learned my addr 172.21.15.62:0/829671082 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b259c640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 0x55c3910fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 -- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 msgr2=0x55c3910fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 0x55c3910fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 -- 172.21.15.62:0/829671082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910eac20 msgr2=0x55c3911640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910eac20 0x55c3911640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 -- 172.21.15.62:0/829671082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c391164680 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b259c640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 0x55c3910fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f55b1d9b640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006a70 tx=0x7f55a8009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cddaf9c28f3f321 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.319+0000 7f5599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a80099b0 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55a8019960 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a8032070 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55a8045050 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3911658f0 con 0x55c3910db510 2024-08-16T19:10:47.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8032070 con 0x55c3910db510 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f55a8018cd0 con 0x55c3910db510 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 10 client.15483 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 20 client.15483 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c391169c50 0x55c391168960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 4 client.15483 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.320+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 --> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] -- command(tid 0: {"prefix": "status"}) -- 0x55c391048190 con 0x55c391169c50 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55b259c640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c391169c50 0x55c391168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55b259c640 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c391169c50 0x55c391168960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42e8f07f0a67ccf1 server_cookie=39e751001b516a57 in_seq=0 out_seq=0 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f5599ffb640 10 client.15483 ms_handle_connect on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f5599ffb640 1 -- 172.21.15.62:0/829671082 <== mds.2 v2:172.21.15.161:6836/3769663917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c391048190 con 0x55c391169c50 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f5599ffb640 10 client.15483 handle_command_reply: tid=0 2024-08-16T19:10:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 client.15483 shutdown 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 msgr2=0x55c3910fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006a70 tx=0x7f55a8009420 comp rx=0 tx=0).stop 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 shutdown_connections 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:6836/3769663917,v1:172.21.15.161:6837/3769663917] conn(0x55c391169c50 0x55c391168960 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3910eac20 0x55c3911640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c3910dbe30 0x55c3910fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 --2- 172.21.15.62:0/829671082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c3910db510 0x55c3910fe590 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f5599ffb640 0 client.15483 ms_handle_reset on v2:172.21.15.161:6836/3769663917 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.321+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 >> 172.21.15.62:0/829671082 conn(0x55c3910d7bc0 msgr2=0x55c3910e6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 shutdown_connections 2024-08-16T19:10:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55bb113740 1 -- 172.21.15.62:0/829671082 wait complete. 2024-08-16T19:10:47.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55bb113740 20 client.15483 trim_cache size 0 max 16384 2024-08-16T19:10:47.328 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:47.328 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:47.328 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 2, 2024-08-16T19:10:47.328 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24451, 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 121.58480332000001, 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 28, 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 93, 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 92, 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 187.029524823 2024-08-16T19:10:47.329 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 msgr2=0x7f559c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 0x7f559c0a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0cd3d0 tx=0x7f55ac06a040 comp rx=0 tx=0).stop 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 msgr2=0x7f55b419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00cce0 tx=0x7f55ac00a620 comp rx=0 tx=0).stop 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 shutdown_connections 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f559c0a2d80 0x7f559c0a5240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55b410bea0 0x7f55b419ca70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55b4103390 0x7f55b41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 --2- 172.21.15.62:0/1013592357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55b4102a70 0x7f55b41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 >> 172.21.15.62:0/1013592357 conn(0x7f55b40fc4d0 msgr2=0x7f55b40fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 shutdown_connections 2024-08-16T19:10:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:47.322+0000 7f55b8d15640 1 -- 172.21.15.62:0/1013592357 wait complete. 2024-08-16T19:10:47.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:47 smithi062 ceph-mon[32993]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 767 B/s wr, 21 op/s 2024-08-16T19:10:47.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:47 smithi161 ceph-mon[37576]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 767 B/s wr, 21 op/s 2024-08-16T19:10:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:47 smithi155 ceph-mon[37413]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 767 B/s wr, 21 op/s 2024-08-16T19:10:47.989 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-16T19:10:48.252 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 -- 172.21.15.62:0/913926460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c102a70 msgr2=0x7f545c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 --2- 172.21.15.62:0/913926460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c102a70 0x7f545c102e50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c02a480 comp rx=0 tx=0).stop 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 -- 172.21.15.62:0/913926460 shutdown_connections 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 --2- 172.21.15.62:0/913926460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c10bea0 0x7f545c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 --2- 172.21.15.62:0/913926460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c103390 0x7f545c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 --2- 172.21.15.62:0/913926460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c102a70 0x7f545c102e50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 -- 172.21.15.62:0/913926460 >> 172.21.15.62:0/913926460 conn(0x7f545c0fc4d0 msgr2=0x7f545c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.746+0000 7f5461eff640 1 -- 172.21.15.62:0/913926460 shutdown_connections 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 -- 172.21.15.62:0/913926460 wait complete. 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 Processor -- start 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 -- start start 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c102a70 0x7f545c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c103390 0x7f545c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f545c111cf0 con 0x7f545c10bea0 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f545c111b70 con 0x7f545c103390 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.747+0000 7f5461eff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f545c111e70 con 0x7f545c102a70 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42488/0 (socket says 172.21.15.62:42488) 2024-08-16T19:10:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 -- 172.21.15.62:0/2156181349 learned_addr learned my addr 172.21.15.62:0/2156181349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c102a70 0x7f545c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545affd640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c103390 0x7f545c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 -- 172.21.15.62:0/2156181349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c102a70 msgr2=0x7f545c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c102a70 0x7f545c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 -- 172.21.15.62:0/2156181349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c103390 msgr2=0x7f545c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c103390 0x7f545c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 -- 172.21.15.62:0/2156181349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19d150 con 0x7f545c10bea0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f545bfff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f545400cad0 tx=0x7f545400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f5458ff9640 1 -- 172.21.15.62:0/2156181349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400ae20 con 0x7f545c10bea0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f5458ff9640 1 -- 172.21.15.62:0/2156181349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5454005020 con 0x7f545c10bea0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f5458ff9640 1 -- 172.21.15.62:0/2156181349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545400f6f0 con 0x7f545c10bea0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19d440 con 0x7f545c10bea0 2024-08-16T19:10:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.748+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a4d20 con 0x7f545c10bea0 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f5458ff9640 1 -- 172.21.15.62:0/2156181349 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f54540051d0 con 0x7f545c10bea0 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 Processor -- start 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 -- start start 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd6277e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd62826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562dd61d8710 con 0x562dd6282c20 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dd6170770 con 0x562dd6277a40 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.751+0000 7f54642fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562dd61d46d0 con 0x562dd6278360 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f5458ff9640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 0x7f54440a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42498/0 (socket says 172.21.15.62:42498) 2024-08-16T19:10:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 -- 172.21.15.62:0/2450822949 learned_addr learned my addr 172.21.15.62:0/2450822949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd62826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd6277e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545bfff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 0x7f54440a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 -- 172.21.15.62:0/2450822949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 msgr2=0x562dd62826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd62826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 -- 172.21.15.62:0/2450822949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 msgr2=0x562dd6277e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f5458ff9640 1 -- 172.21.15.62:0/2156181349 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f54540cf9b0 con 0x7f545c10bea0 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd6277e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 -- 172.21.15.62:0/2450822949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd61773e0 con 0x562dd6282c20 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545affd640 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f54500103a0 tx=0x7f5450008510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b104f4cbb4b5a817 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f5441ffb640 1 -- 172.21.15.62:0/2450822949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450005020 con 0x562dd6282c20 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f5441ffb640 1 -- 172.21.15.62:0/2450822949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54500047a0 con 0x562dd6282c20 2024-08-16T19:10:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f545bfff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 0x7f54440a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54540ccd10 tx=0x7f5454086040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.752+0000 7f5441ffb640 1 -- 172.21.15.62:0/2450822949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500128c0 con 0x562dd6282c20 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- 172.21.15.62:0/2450822949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 msgr2=0x562dd6285010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f54500103a0 tx=0x7f5450008510 comp rx=0 tx=0).stop 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- 172.21.15.62:0/2450822949 shutdown_connections 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6285010 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd62826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 --2- 172.21.15.62:0/2450822949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd6277e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- 172.21.15.62:0/2450822949 >> 172.21.15.62:0/2450822949 conn(0x562dd6273fd0 msgr2=0x562dd62743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- 172.21.15.62:0/2450822949 shutdown_connections 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- 172.21.15.62:0/2450822949 wait complete. 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.753+0000 7f54642fd740 1 -- start start 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 Processor -- start 2024-08-16T19:10:48.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd627fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd6280040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562dd62857f0 con 0x562dd6282c20 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dd6193970 con 0x562dd6277a40 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562dd6285970 con 0x562dd6278360 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f54642fd740 10 client.0 _ll_register_callbacks cb 0x562dd5ee77b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42504/0 (socket says 172.21.15.62:42504) 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 -- 172.21.15.62:0/1525718291 learned_addr learned my addr 172.21.15.62:0/1525718291 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd627fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545b7fe640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd6280040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 -- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 msgr2=0x562dd6280040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd6280040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 -- 172.21.15.62:0/1525718291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 msgr2=0x562dd627fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd627fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 -- 172.21.15.62:0/1525718291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd5f4a2b0 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545b7fe640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd6280040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.754+0000 7f545affd640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5450001eb0 tx=0x7f5450008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4fc68b2da985f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450023070 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54500047a0 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f5450012660 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5450031050 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 10 client.15495 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562dd62913f0 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54500047a0 con 0x562dd6282c20 2024-08-16T19:10:48.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f545001f720 con 0x562dd6282c20 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 10 client.15495 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 20 client.15495 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd62937e0 0x562dd6294c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 4 client.15495 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.755+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dd61e5c80 con 0x562dd62937e0 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.756+0000 7f545b7fe640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd62937e0 0x562dd6294c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.756+0000 7f545b7fe640 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd62937e0 0x562dd6294c50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93c68c16d116a6c6 server_cookie=18cd56fabeddf0a6 in_seq=0 out_seq=0 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.756+0000 7f543affd640 10 client.15495 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f543affd640 1 -- 172.21.15.62:0/1525718291 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562dd61e5c80 con 0x562dd62937e0 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f543affd640 10 client.15495 handle_command_reply: tid=0 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 client.15495 shutdown 2024-08-16T19:10:48.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 msgr2=0x562dd6280580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5450001eb0 tx=0x7f5450008780 comp rx=0 tx=0).stop 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 shutdown_connections 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd62937e0 0x562dd6294c50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6282c20 0x562dd6280580 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6278360 0x562dd6280040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f543affd640 0 client.15495 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 --2- 172.21.15.62:0/1525718291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd627fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 >> 172.21.15.62:0/1525718291 conn(0x562dd6273fd0 msgr2=0x562dd626ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 shutdown_connections 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.757+0000 7f54642fd740 1 -- 172.21.15.62:0/1525718291 wait complete. 2024-08-16T19:10:48.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.758+0000 7f54642fd740 20 client.15495 trim_cache size 0 max 16384 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 Processor -- start 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 -- start start 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 0x562dd627f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd628e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562dd6292f90 con 0x562dd6277a40 2024-08-16T19:10:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dd6293660 con 0x562dd6278360 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f54642fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562dd6293d30 con 0x562dd6282c20 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42506/0 (socket says 172.21.15.62:42506) 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 -- 172.21.15.62:0/2874835973 learned_addr learned my addr 172.21.15.62:0/2874835973 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 -- 172.21.15.62:0/2874835973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 msgr2=0x562dd628e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd628e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 0x562dd627f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 -- 172.21.15.62:0/2874835973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 msgr2=0x562dd627f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 0x562dd627f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545affd640 1 -- 172.21.15.62:0/2874835973 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd6273e00 con 0x562dd6277a40 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.776+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 0x562dd627f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f545affd640 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5450005ad0 tx=0x7f54500048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f76559886031fb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f54417fa640 1 -- 172.21.15.62:0/2874835973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450023040 con 0x562dd6277a40 2024-08-16T19:10:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f54417fa640 1 -- 172.21.15.62:0/2874835973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54500084f0 con 0x562dd6277a40 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f54417fa640 1 -- 172.21.15.62:0/2874835973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545001f6e0 con 0x562dd6277a40 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f54642fd740 1 -- 172.21.15.62:0/2874835973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 msgr2=0x562dd6282260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.777+0000 7f54642fd740 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5450005ad0 tx=0x7f54500048a0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- 172.21.15.62:0/2874835973 shutdown_connections 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd628e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6278360 0x562dd627f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- 172.21.15.62:0/2874835973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6277a40 0x562dd6282260 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- 172.21.15.62:0/2874835973 >> 172.21.15.62:0/2874835973 conn(0x562dd6273fd0 msgr2=0x562dd628dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- 172.21.15.62:0/2874835973 shutdown_connections 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- 172.21.15.62:0/2874835973 wait complete. 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- start start 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 Processor -- start 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd629d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd627cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562dd6294ad0 con 0x562dd6278360 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x562dd6294400 con 0x562dd6277a40 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562dd62951a0 con 0x562dd6282c20 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.778+0000 7f54642fd740 10 client.0 _ll_register_callbacks cb 0x7f5450008e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42508/0 (socket says 172.21.15.62:42508) 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 -- 172.21.15.62:0/2416619612 learned_addr learned my addr 172.21.15.62:0/2416619612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545affd640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd629d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545affd640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd627cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 -- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 msgr2=0x562dd627cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd627cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 -- 172.21.15.62:0/2416619612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 msgr2=0x562dd629d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd629d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 -- 172.21.15.62:0/2416619612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd6274c60 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545affd640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd627cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f545b7fe640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f544c005ab0 tx=0x7f544c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9827f8dc702090f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f543affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c046030 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f544c005020 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54500084f0 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f544c052050 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f54642fd740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.779+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562dd626b130 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54500084f0 con 0x562dd6278360 2024-08-16T19:10:48.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f544c004830 con 0x562dd6278360 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 10 client.15501 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 20 client.15501 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd63024c0 0x562dd63011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 4 client.15501 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 --> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] -- command(tid 0: {"prefix": "status"}) -- 0x562dd61e0190 con 0x562dd63024c0 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.780+0000 7f545affd640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd63024c0 0x562dd63011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f545affd640 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd63024c0 0x562dd63011d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba4ea4a118fd077b server_cookie=4d05cd7a3bd428de in_seq=0 out_seq=0 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f543affd640 10 client.15501 ms_handle_connect on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f543affd640 1 -- 172.21.15.62:0/2416619612 <== mds.1 v2:172.21.15.161:6838/2650096635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562dd61e0190 con 0x562dd63024c0 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f543affd640 10 client.15501 handle_command_reply: tid=0 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 client.15501 shutdown 2024-08-16T19:10:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 msgr2=0x562dd627c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f544c005ab0 tx=0x7f544c0044b0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 shutdown_connections 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:6838/2650096635,v1:172.21.15.161:6839/2650096635] conn(0x562dd63024c0 0x562dd63011d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562dd6282c20 0x562dd627cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562dd6278360 0x562dd627c6e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 --2- 172.21.15.62:0/2416619612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562dd6277a40 0x562dd629d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f543affd640 0 client.15501 ms_handle_reset on v2:172.21.15.161:6838/2650096635 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 >> 172.21.15.62:0/2416619612 conn(0x562dd6273fd0 msgr2=0x562dd626ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 shutdown_connections 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 1 -- 172.21.15.62:0/2416619612 wait complete. 2024-08-16T19:10:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.781+0000 7f54642fd740 20 client.15501 trim_cache size 0 max 16384 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 1, 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24469, 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 124.056112263, 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 28, 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 93, 2024-08-16T19:10:48.794 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 92, 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 183.216821904 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 msgr2=0x7f54440a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 0x7f54440a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54540ccd10 tx=0x7f5454086040 comp rx=0 tx=0).stop 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 msgr2=0x7f545c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:48.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f545400cad0 tx=0x7f545400cf90 comp rx=0 tx=0).stop 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 shutdown_connections 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f54440a2e00 0x7f54440a52c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f545c10bea0 0x7f545c19ca70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f545c103390 0x7f545c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 --2- 172.21.15.62:0/2156181349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f545c102a70 0x7f545c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 >> 172.21.15.62:0/2156181349 conn(0x7f545c0fc4d0 msgr2=0x7f545c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 shutdown_connections 2024-08-16T19:10:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:48.782+0000 7f5461eff640 1 -- 172.21.15.62:0/2156181349 wait complete. 2024-08-16T19:10:49.398 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-16T19:10:49.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:49 smithi161 ceph-mon[37576]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:49.668 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:10:49.696 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:49 smithi062 ceph-mon[32993]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:49.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:49 smithi155 ceph-mon[37413]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:50.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 -- 172.21.15.62:0/3927036986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0103410 msgr2=0x7f75a01078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/3927036986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0103410 0x7f75a01078c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7594009930 tx=0x7f759402a420 comp rx=0 tx=0).stop 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 -- 172.21.15.62:0/3927036986 shutdown_connections 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/3927036986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0107fd0 0x7f75a010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/3927036986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0103410 0x7f75a01078c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/3927036986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 0x7f75a0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 -- 172.21.15.62:0/3927036986 >> 172.21.15.62:0/3927036986 conn(0x7f75a00fc500 msgr2=0x7f75a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 -- 172.21.15.62:0/3927036986 shutdown_connections 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 -- 172.21.15.62:0/3927036986 wait complete. 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.234+0000 7f75a7dbf640 1 Processor -- start 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 -- start start 2024-08-16T19:10:50.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 0x7f75a01006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0103410 0x7f75a0100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0111dd0 con 0x7f75a0107fd0 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0111c50 con 0x7f75a0102a60 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a7dbf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0111f50 con 0x7f75a0103410 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42528/0 (socket says 172.21.15.62:42528) 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 -- 172.21.15.62:0/2267572880 learned_addr learned my addr 172.21.15.62:0/2267572880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 0x7f75a01006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a535a640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0103410 0x7f75a0100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 -- 172.21.15.62:0/2267572880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0103410 msgr2=0x7f75a0100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0103410 0x7f75a0100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 -- 172.21.15.62:0/2267572880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 msgr2=0x7f75a01006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 0x7f75a01006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.235+0000 7f75a635c640 1 -- 172.21.15.62:0/2267572880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a01a6830 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f75a635c640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f759800eaa0 tx=0x7f759800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f7592ffd640 1 -- 172.21.15.62:0/2267572880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800cdb0 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a01a6b20 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f7592ffd640 1 -- 172.21.15.62:0/2267572880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598005020 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f7592ffd640 1 -- 172.21.15.62:0/2267572880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75980116b0 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.236+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a01a7030 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.239+0000 7f75aa1bd740 1 Processor -- start 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.239+0000 7f7592ffd640 1 -- 172.21.15.62:0/2267572880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f7598005560 con 0x7f75a0107fd0 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.239+0000 7f75aa1bd740 1 -- start start 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab122a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 0x560aab12d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560aab087710 con 0x560aab12d810 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560aab01f770 con 0x560aab122630 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560aab0836d0 con 0x560aab122f50 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a535a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a535a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42538/0 (socket says 172.21.15.62:42538) 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a535a640 1 -- 172.21.15.62:0/930889713 learned_addr learned my addr 172.21.15.62:0/930889713 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 0x560aab12d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a535a640 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f7592ffd640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 0x7f758c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 -- 172.21.15.62:0/930889713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 msgr2=0x560aab122a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab122a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 -- 172.21.15.62:0/930889713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 msgr2=0x560aab12fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a5b5b640 1 -- 172.21.15.62:0/930889713 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aab0263e0 con 0x560aab122f50 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.240+0000 7f75a635c640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 0x7f758c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f7592ffd640 1 -- 172.21.15.62:0/2267572880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f75980cfab0 con 0x7f75a0107fd0 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 0x560aab12d2d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f759c005fd0 tx=0x7f759c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ebbe4d3686abd97 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f75a635c640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 0x7f758c0a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75980cce10 tx=0x7f759804b040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f758bfff640 1 -- 172.21.15.62:0/930889713 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c01c480 con 0x560aab122f50 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f758bfff640 1 -- 172.21.15.62:0/930889713 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c01cb00 con 0x560aab122f50 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.241+0000 7f758bfff640 1 -- 172.21.15.62:0/930889713 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c011b90 con 0x560aab122f50 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 -- 172.21.15.62:0/930889713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 msgr2=0x560aab12d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 0x560aab12d2d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f759c005fd0 tx=0x7f759c0107a0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 -- 172.21.15.62:0/930889713 shutdown_connections 2024-08-16T19:10:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122f50 0x560aab12d2d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/930889713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab122a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 -- 172.21.15.62:0/930889713 >> 172.21.15.62:0/930889713 conn(0x560aab11ebc0 msgr2=0x560aab11efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 -- 172.21.15.62:0/930889713 shutdown_connections 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.242+0000 7f75aa1bd740 1 -- 172.21.15.62:0/930889713 wait complete. 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 -- start start 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 Processor -- start 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab12a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab12d810 0x560aab12b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560aab1303e0 con 0x560aab122f50 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560aab042970 con 0x560aab122630 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560aab130560 con 0x560aab12d810 2024-08-16T19:10:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75aa1bd740 10 client.0 _ll_register_callbacks cb 0x560aaad967b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75a5b5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75a5b5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42552/0 (socket says 172.21.15.62:42552) 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.243+0000 7f75a5b5b640 1 -- 172.21.15.62:0/4264419514 learned_addr learned my addr 172.21.15.62:0/4264419514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a535a640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab12a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a535a640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab12d810 0x560aab12b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 -- 172.21.15.62:0/4264419514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab12d810 msgr2=0x560aab12b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab12d810 0x560aab12b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 -- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 msgr2=0x560aab12a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab12a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 -- 172.21.15.62:0/4264419514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaadf92b0 con 0x560aab122f50 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f759c00bf50 tx=0x7f759c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3cc4ed765a3ba9 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c02b060 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f759c01c690 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759c01cb00 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f759c032050 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75aa1bd740 10 client.15513 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.244+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560aab13c040 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c01cb00 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f759c0267a0 con 0x560aab122f50 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 10 client.15513 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 20 client.15513 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab13e2e0 0x560aab13f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 4 client.15513 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:10:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aab094c80 con 0x560aab13e2e0 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.245+0000 7f75a535a640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab13e2e0 0x560aab13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.246+0000 7f75897fa640 10 client.15513 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.246+0000 7f75a535a640 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab13e2e0 0x560aab13f750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20d816c81254667 server_cookie=b5dea51a80db10e7 in_seq=0 out_seq=0 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75897fa640 1 -- 172.21.15.62:0/4264419514 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560aab094c80 con 0x560aab13e2e0 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75897fa640 10 client.15513 handle_command_reply: tid=0 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 client.15513 shutdown 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 msgr2=0x560aab12ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f759c00bf50 tx=0x7f759c005740 comp rx=0 tx=0).stop 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 shutdown_connections 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab13e2e0 0x560aab13f750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab12d810 0x560aab12b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab12ac60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/4264419514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122630 0x560aab12a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75897fa640 0 client.15513 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 >> 172.21.15.62:0/4264419514 conn(0x560aab11ebc0 msgr2=0x560aab11dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 shutdown_connections 2024-08-16T19:10:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 1 -- 172.21.15.62:0/4264419514 wait complete. 2024-08-16T19:10:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.247+0000 7f75aa1bd740 20 client.15513 trim_cache size 0 max 16384 2024-08-16T19:10:50.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.277+0000 7f75aa1bd740 1 Processor -- start 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.277+0000 7f75aa1bd740 1 -- start start 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab13f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 0x560aab125090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560aab13da90 con 0x560aab12d810 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560aab13e160 con 0x560aab122f50 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560aab13e830 con 0x560aab122630 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42566/0 (socket says 172.21.15.62:42566) 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 -- 172.21.15.62:0/3878171723 learned_addr learned my addr 172.21.15.62:0/3878171723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 0x560aab125090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 -- 172.21.15.62:0/3878171723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 msgr2=0x560aab13f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab13f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 -- 172.21.15.62:0/3878171723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 msgr2=0x560aab125090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 0x560aab125090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 -- 172.21.15.62:0/3878171723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aab122460 con 0x560aab12d810 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 0x560aab125090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.278+0000 7f75a535a640 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7594009930 tx=0x7f7594005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ee204c055cf6a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f758b7fe640 1 -- 172.21.15.62:0/3878171723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594035040 con 0x560aab12d810 2024-08-16T19:10:50.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f758b7fe640 1 -- 172.21.15.62:0/3878171723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75940035f0 con 0x560aab12d810 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f758b7fe640 1 -- 172.21.15.62:0/3878171723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759404c5a0 con 0x560aab12d810 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 -- 172.21.15.62:0/3878171723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 msgr2=0x560aab12aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7594009930 tx=0x7f7594005780 comp rx=0 tx=0).stop 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 -- 172.21.15.62:0/3878171723 shutdown_connections 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab12d810 0x560aab12aff0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab122f50 0x560aab125090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/3878171723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 -- 172.21.15.62:0/3878171723 >> 172.21.15.62:0/3878171723 conn(0x560aab11ebc0 msgr2=0x560aab12c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.279+0000 7f75aa1bd740 1 -- 172.21.15.62:0/3878171723 shutdown_connections 2024-08-16T19:10:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 -- 172.21.15.62:0/3878171723 wait complete. 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 -- start start 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 Processor -- start 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 0x560aab146420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560aab13f5d0 con 0x560aab122f50 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x560aab13ef00 con 0x560aab12d810 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560aab13fca0 con 0x560aab122630 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75aa1bd740 10 client.0 _ll_register_callbacks cb 0x7f7594009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.280+0000 7f75a5b5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42580/0 (socket says 172.21.15.62:42580) 2024-08-16T19:10:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 -- 172.21.15.62:0/1177737102 learned_addr learned my addr 172.21.15.62:0/1177737102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a535a640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a535a640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 0x560aab146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 -- 172.21.15.62:0/1177737102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 msgr2=0x560aab141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 -- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 msgr2=0x560aab146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 0x560aab146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 -- 172.21.15.62:0/1177737102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aab11a130 con 0x560aab122f50 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a535a640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 0x560aab146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75a5b5b640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f759c000c00 tx=0x7f759c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745213bc9fd4e98c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75897fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c004020 con 0x560aab122f50 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f759c022070 con 0x560aab122f50 2024-08-16T19:10:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75940035f0 con 0x560aab122f50 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f759c043050 con 0x560aab122f50 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75aa1bd740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.281+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560aab1a9030 con 0x560aab122f50 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75940035f0 con 0x560aab122f50 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f759c005200 con 0x560aab122f50 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 10 client.15519 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 20 client.15519 populate_metadata read hostname 'smithi062' 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab1adb10 0x560aab1ac820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 4 client.15519 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "status"}) -- 0x560aab08f190 con 0x560aab1adb10 2024-08-16T19:10:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.282+0000 7f75a535a640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab1adb10 0x560aab1ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:10:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75a535a640 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab1adb10 0x560aab1ac820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b397e3f623918c84 server_cookie=c2e3ea9376eea189 in_seq=0 out_seq=0 2024-08-16T19:10:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75897fa640 10 client.15519 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:50.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75897fa640 1 -- 172.21.15.62:0/1177737102 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560aab08f190 con 0x560aab1adb10 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75897fa640 10 client.15519 handle_command_reply: tid=0 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 client.15519 shutdown 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 msgr2=0x560aab142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f759c000c00 tx=0x7f759c0057b0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 shutdown_connections 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x560aab1adb10 0x560aab1ac820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560aab12d810 0x560aab146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560aab122f50 0x560aab142510 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 --2- 172.21.15.62:0/1177737102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560aab122630 0x560aab141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75897fa640 0 client.15519 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 >> 172.21.15.62:0/1177737102 conn(0x560aab11ebc0 msgr2=0x560aab12bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 shutdown_connections 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 1 -- 172.21.15.62:0/1177737102 wait complete. 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.283+0000 7f75aa1bd740 20 client.15519 trim_cache size 0 max 16384 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "bd001ebc-5c01-11ef-bccf-c7b262605968", 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 24476, 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-08-16T19:10:50.291 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 164.70595127600001, 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 28, 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 93, 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 92, 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 175.744124156 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:10:50.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 msgr2=0x7f758c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 0x7f758c0a5240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75980cce10 tx=0x7f759804b040 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 msgr2=0x7f75a0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f759800eaa0 tx=0x7f759800ef60 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 shutdown_connections 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f758c0a2d80 0x7f758c0a5240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75a0107fd0 0x7f75a0101140 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75a0103410 0x7f75a0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 --2- 172.21.15.62:0/2267572880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75a0102a60 0x7f75a01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 >> 172.21.15.62:0/2267572880 conn(0x7f75a00fc500 msgr2=0x7f75a01060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 shutdown_connections 2024-08-16T19:10:50.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:10:50.284+0000 7f75a7dbf640 1 -- 172.21.15.62:0/2267572880 wait complete. 2024-08-16T19:10:50.923 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-16T19:10:51.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:51 smithi161 ceph-mon[37576]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-08-16T19:10:51.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:51 smithi062 ceph-mon[32993]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-08-16T19:10:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:51 smithi155 ceph-mon[37413]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-08-16T19:10:53.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:53 smithi161 ceph-mon[37576]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:53 smithi062 ceph-mon[32993]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:53 smithi155 ceph-mon[37413]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-16T19:10:55.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:55 smithi161 ceph-mon[37576]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-16T19:10:55.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:55 smithi062 ceph-mon[32993]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-16T19:10:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:55 smithi155 ceph-mon[37413]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 11 op/s 2024-08-16T19:10:57.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:57 smithi161 ceph-mon[37576]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-16T19:10:57.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:57 smithi062 ceph-mon[32993]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-16T19:10:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:57 smithi155 ceph-mon[37413]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-16T19:10:59.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:10:59 smithi161 ceph-mon[37576]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:10:59.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:10:59 smithi062 ceph-mon[32993]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:10:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:10:59 smithi155 ceph-mon[37413]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:01.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:11:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:11:00.810+0000 7feda6fe5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:11:01.246 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:11:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:11:00.810+0000 7feda6fe5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:11:01.247 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:11:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:11:00.810+0000 7feda6fe5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:11:01.247 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:11:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:11:00.810+0000 7feda6fe5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:11:01.247 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:11:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:11:00.810+0000 7feda6fe5640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-16T19:11:01.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:01 smithi161 ceph-mon[37576]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:01 smithi062 ceph-mon[32993]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:01 smithi155 ceph-mon[37413]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:03 smithi062 ceph-mon[32993]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:03 smithi062 ceph-mon[32993]: mgrmap e24: x(active, since 34s), standbys: y, z 2024-08-16T19:11:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:03 smithi155 ceph-mon[37413]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:03 smithi155 ceph-mon[37413]: mgrmap e24: x(active, since 34s), standbys: y, z 2024-08-16T19:11:03.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:03 smithi161 ceph-mon[37576]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:03.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:03 smithi161 ceph-mon[37576]: mgrmap e24: x(active, since 34s), standbys: y, z 2024-08-16T19:11:05.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:05 smithi062 ceph-mon[32993]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:05.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:05 smithi155 ceph-mon[37413]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:05.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:05 smithi161 ceph-mon[37576]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:05.924 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.6 2024-08-16T19:11:05.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:11:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:11:07.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:07 smithi062 ceph-mon[32993]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:07 smithi155 ceph-mon[37413]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:07 smithi161 ceph-mon[37576]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-08-16T19:11:08.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 -- 172.21.15.62:0/4125433215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc1022a0 msgr2=0x7f9afc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:08.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 --2- 172.21.15.62:0/4125433215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc1022a0 0x7f9afc1026e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009900 tx=0x7f9af002a480 comp rx=0 tx=0).stop 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 -- 172.21.15.62:0/4125433215 shutdown_connections 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 --2- 172.21.15.62:0/4125433215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc102c20 0x7f9afc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 --2- 172.21.15.62:0/4125433215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc1022a0 0x7f9afc1026e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 --2- 172.21.15.62:0/4125433215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc100b50 0x7f9afc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 -- 172.21.15.62:0/4125433215 >> 172.21.15.62:0/4125433215 conn(0x7f9afc0fc500 msgr2=0x7f9afc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 -- 172.21.15.62:0/4125433215 shutdown_connections 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.297+0000 7f9b019ef640 1 -- 172.21.15.62:0/4125433215 wait complete. 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 Processor -- start 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 -- start start 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc1022a0 0x7f9afc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:08.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 0x7f9afc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc110cb0 con 0x7f9afc102c20 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc110b30 con 0x7f9afc1022a0 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.298+0000 7f9b019ef640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9afc110e30 con 0x7f9afc100b50 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 0x7f9afc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:52986/0 (socket says 172.21.15.62:52986) 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 -- 172.21.15.62:0/1222614431 learned_addr learned my addr 172.21.15.62:0/1222614431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afa7fc640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc1022a0 0x7f9afc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 -- 172.21.15.62:0/1222614431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc1022a0 msgr2=0x7f9afc1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc1022a0 0x7f9afc1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 -- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 msgr2=0x7f9afc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 0x7f9afc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 -- 172.21.15.62:0/1222614431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc19d130 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afb7fe640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 0x7f9afc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.299+0000 7f9afaffd640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c920 tx=0x7f9aec00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.300+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007c70 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.300+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc19d420 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.300+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec007e10 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.300+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec018890 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.300+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9afc1a4d00 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.301+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc100f30 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.302+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f9aec018a50 con 0x7f9afc100b50 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.303+0000 7f9b009ed640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 0x7f9ae40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:08.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.303+0000 7f9afa7fc640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 0x7f9ae40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.303+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f9aec0cfd70 con 0x7f9afc100b50 2024-08-16T19:11:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.304+0000 7f9afa7fc640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 0x7f9ae40a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009230 tx=0x7f9af00058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.306+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9aec014030 con 0x7f9afc100b50 2024-08-16T19:11:08.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.522+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9afc1994e0 con 0x7f9afc100b50 2024-08-16T19:11:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.523+0000 7f9b009ed640 1 -- 172.21.15.62:0/1222614431 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f9aec09c4c0 con 0x7f9afc100b50 2024-08-16T19:11:08.523 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:11:08.524 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-08-16T19:10:26:918579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:10:26.868424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:11:08.524 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-08-16T19:11:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 msgr2=0x7f9ae40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 0x7f9ae40a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009230 tx=0x7f9af00058d0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 msgr2=0x7f9afc198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:08.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c920 tx=0x7f9aec00cde0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 shutdown_connections 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9ae40a2e90 0x7f9ae40a5350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9afc102c20 0x7f9afc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9afc1022a0 0x7f9afc1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 --2- 172.21.15.62:0/1222614431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9afc100b50 0x7f9afc198180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 >> 172.21.15.62:0/1222614431 conn(0x7f9afc0fc500 msgr2=0x7f9afc069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 shutdown_connections 2024-08-16T19:11:08.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:08.525+0000 7f9b019ef640 1 -- 172.21.15.62:0/1222614431 wait complete. 2024-08-16T19:11:09.239 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 143.6 --await-for 45.0' 2024-08-16T19:11:09.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 143.6 --await-for 45.0 2024-08-16T19:11:09.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:09 smithi062 ceph-mon[32993]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-16T19:11:09.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:09 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1222614431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:09.536 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:11:09.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:09 smithi155 ceph-mon[37413]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-16T19:11:09.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:09 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1222614431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:09 smithi161 ceph-mon[37576]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-08-16T19:11:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:09 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1222614431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 -- 172.21.15.62:0/748595146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 msgr2=0x7eff64102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 --2- 172.21.15.62:0/748595146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64102e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009cd0 tx=0x7eff5c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 -- 172.21.15.62:0/748595146 shutdown_connections 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 --2- 172.21.15.62:0/748595146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 0x7eff6410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 --2- 172.21.15.62:0/748595146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 0x7eff641078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 --2- 172.21.15.62:0/748595146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64102e40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 -- 172.21.15.62:0/748595146 >> 172.21.15.62:0/748595146 conn(0x7eff640fc500 msgr2=0x7eff640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 -- 172.21.15.62:0/748595146 shutdown_connections 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.032+0000 7eff6c54a640 1 -- 172.21.15.62:0/748595146 wait complete. 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 Processor -- start 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 -- start start 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 0x7eff64198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 0x7eff6419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111dd0 con 0x7eff64102a60 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111c50 con 0x7eff64107fd0 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6c54a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111f50 con 0x7eff64103410 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6a2e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6a2e6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43982/0 (socket says 172.21.15.62:43982) 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.033+0000 7eff6a2e6640 1 -- 172.21.15.62:0/1426141221 learned_addr learned my addr 172.21.15.62:0/1426141221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff69ae5640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 0x7eff64198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6aae7640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 0x7eff6419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 -- 172.21.15.62:0/1426141221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 msgr2=0x7eff64198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 0x7eff64198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 -- 172.21.15.62:0/1426141221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 msgr2=0x7eff6419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 0x7eff6419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 -- 172.21.15.62:0/1426141221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6419d2c0 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff637fe640 1 -- 172.21.15.62:0/1426141221 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004040 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff637fe640 1 -- 172.21.15.62:0/1426141221 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c0046c0 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6419d550 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff637fe640 1 -- 172.21.15.62:0/1426141221 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c046480 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.034+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a4e90 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.037+0000 7eff637fe640 1 -- 172.21.15.62:0/1426141221 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7eff5c0305a0 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.037+0000 7eff637fe640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 0x7eff4c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.037+0000 7eff637fe640 1 -- 172.21.15.62:0/1426141221 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7eff5c02e030 con 0x7eff64102a60 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.037+0000 7eff69ae5640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 0x7eff4c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 Processor -- start 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff69ae5640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 0x7eff4c0a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff640feba0 tx=0x7eff58005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 -- start start 2024-08-16T19:11:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fcd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fcda50 0x55b406fd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b406f32a70 con 0x55b406fd8310 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b406eca780 con 0x55b406fcda50 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.038+0000 7eff6e948740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b406f2ea30 con 0x55b406fcd130 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43998/0 (socket says 172.21.15.62:43998) 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 -- 172.21.15.62:0/3002034425 learned_addr learned my addr 172.21.15.62:0/3002034425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fcd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff69ae5640 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fcda50 0x55b406fd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 -- 172.21.15.62:0/3002034425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 msgr2=0x55b406fcd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fcd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 -- 172.21.15.62:0/3002034425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fcda50 msgr2=0x55b406fd7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fcda50 0x55b406fd7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 -- 172.21.15.62:0/3002034425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b406ca4b90 con 0x55b406fd8310 2024-08-16T19:11:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff6aae7640 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff54009ab0 tx=0x7eff54009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d3df356478fa29 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff60ff9640 1 -- 172.21.15.62:0/3002034425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54016070 con 0x55b406fd8310 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff60ff9640 1 -- 172.21.15.62:0/3002034425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5400ecb0 con 0x55b406fd8310 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.039+0000 7eff60ff9640 1 -- 172.21.15.62:0/3002034425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54010700 con 0x55b406fd8310 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.040+0000 7eff6e948740 1 -- 172.21.15.62:0/3002034425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 msgr2=0x55b406fda700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.040+0000 7eff6e948740 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff54009ab0 tx=0x7eff54009f70 comp rx=0 tx=0).stop 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 -- 172.21.15.62:0/3002034425 shutdown_connections 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fd8310 0x55b406fda700 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fcda50 0x55b406fd7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 --2- 172.21.15.62:0/3002034425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fcd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 -- 172.21.15.62:0/3002034425 >> 172.21.15.62:0/3002034425 conn(0x55b406fc96c0 msgr2=0x55b406fc9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 -- 172.21.15.62:0/3002034425 shutdown_connections 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 -- 172.21.15.62:0/3002034425 wait complete. 2024-08-16T19:11:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 -- start start 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.041+0000 7eff6e948740 1 Processor -- start 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b406fdaee0 con 0x55b406fcda50 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b406eed990 con 0x55b406fd8310 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b406fdb060 con 0x55b406fcd130 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44014/0 (socket says 172.21.15.62:44014) 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 -- 172.21.15.62:0/4071215701 learned_addr learned my addr 172.21.15.62:0/4071215701 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6e948740 10 client.0 _ll_register_callbacks cb 0x55b406c42090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff6aae7640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 -- 172.21.15.62:0/4071215701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 msgr2=0x55b406fd5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 -- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 msgr2=0x55b406fd5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 -- 172.21.15.62:0/4071215701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b406eec6c0 con 0x55b406fcda50 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff69ae5640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff58012c40 tx=0x7eff58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a273735c0980b18 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.042+0000 7eff49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5801fe90 con 0x55b406fcda50 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7eff58011c00 con 0x55b406fcda50 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5400ecb0 con 0x55b406fcda50 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff58011e00 con 0x55b406fcda50 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff6e948740 10 client.15543 fetch_fsmap learned FSMap version 28 2024-08-16T19:11:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b406fe6b70 con 0x55b406fcda50 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5400ecb0 con 0x55b406fcda50 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7eff580205a0 con 0x55b406fcda50 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff6e948740 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.043+0000 7eff6e948740 10 client.15543 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6e948740 20 client.15543 populate_metadata read hostname 'smithi062' 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b406fe8080 0x55b406fea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6e948740 4 client.15543 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b406f401a0 con 0x55b406fe8080 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b406fe8080 0x55b406fea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b406fe8080 0x55b406fea520 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef77e1e391588bd server_cookie=d739c77a47cac62b in_seq=0 out_seq=0 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.044+0000 7eff49ffb640 10 client.15543 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff49ffb640 1 -- 172.21.15.62:0/4071215701 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b406f401a0 con 0x55b406fe8080 2024-08-16T19:11:10.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff49ffb640 10 client.15543 handle_command_reply: tid=0 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 client.15543 shutdown 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 msgr2=0x55b406fd56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff58012c40 tx=0x7eff58004730 comp rx=0 tx=0).stop 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 shutdown_connections 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b406fe8080 0x55b406fea520 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd56b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff49ffb640 0 client.15543 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 --2- 172.21.15.62:0/4071215701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 >> 172.21.15.62:0/4071215701 conn(0x55b406fc96c0 msgr2=0x55b406fcf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 shutdown_connections 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.045+0000 7eff6e948740 1 -- 172.21.15.62:0/4071215701 wait complete. 2024-08-16T19:11:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.046+0000 7eff6e948740 20 client.15543 trim_cache size 0 max 16384 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.064+0000 7eff6e948740 1 Processor -- start 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.064+0000 7eff6e948740 1 -- start start 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fefdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff641aee30 con 0x55b406fcda50 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b0970 con 0x55b406fd8310 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6e948740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff641af500 con 0x55b406fcd130 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44026/0 (socket says 172.21.15.62:44026) 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 -- 172.21.15.62:0/2508098806 learned_addr learned my addr 172.21.15.62:0/2508098806 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fefdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6aae7640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 -- 172.21.15.62:0/2508098806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 msgr2=0x55b406fefdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fefdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 -- 172.21.15.62:0/2508098806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 msgr2=0x55b406fd23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 -- 172.21.15.62:0/2508098806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b406fd4120 con 0x55b406fcda50 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff6aae7640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:11:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.065+0000 7eff69ae5640 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff58012c10 tx=0x7eff58009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963717171cbfd746 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.066+0000 7eff4bfff640 1 -- 172.21.15.62:0/2508098806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58012c70 con 0x55b406fcda50 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.066+0000 7eff4bfff640 1 -- 172.21.15.62:0/2508098806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5803c070 con 0x55b406fcda50 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.066+0000 7eff4bfff640 1 -- 172.21.15.62:0/2508098806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff580035f0 con 0x55b406fcda50 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- 172.21.15.62:0/2508098806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 msgr2=0x55b406ff0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff58012c10 tx=0x7eff58009930 comp rx=0 tx=0).stop 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- 172.21.15.62:0/2508098806 shutdown_connections 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406ff0330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 --2- 172.21.15.62:0/2508098806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fefdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- 172.21.15.62:0/2508098806 >> 172.21.15.62:0/2508098806 conn(0x55b406fc96c0 msgr2=0x55b406fd71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- 172.21.15.62:0/2508098806 shutdown_connections 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- 172.21.15.62:0/2508098806 wait complete. 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 -- start start 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 Processor -- start 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.067+0000 7eff6e948740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b02a0 con 0x55b406fcda50 2024-08-16T19:11:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7eff641afbd0 con 0x55b406fd8310 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff641ace10 con 0x55b406fcd130 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6e948740 10 client.0 _ll_register_callbacks cb 0x7eff58011b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44034/0 (socket says 172.21.15.62:44034) 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 -- 172.21.15.62:0/3969741464 learned_addr learned my addr 172.21.15.62:0/3969741464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff6aae7640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 -- 172.21.15.62:0/3969741464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 msgr2=0x55b406fd2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 -- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 msgr2=0x55b406fd2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 -- 172.21.15.62:0/3969741464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b407051fc0 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff69ae5640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7eff5801ffd0 tx=0x7eff58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d639b610f4dcc025 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58020630 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.068+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7eff580207d0 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5803c070 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff58052050 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b407053240 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5803c070 con 0x55b406fcda50 2024-08-16T19:11:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7eff58049590 con 0x55b406fcda50 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 10 client.15549 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 20 client.15549 populate_metadata read hostname 'smithi062' 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b4070538f0 0x55b407055d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 4 client.15549 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.6, "await_for": 45.0}] 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.069+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.6, "await_for": 45.0}) -- 0x55b406f3a4f0 con 0x55b4070538f0 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.070+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b4070538f0 0x55b407055d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.070+0000 7eff6a2e6640 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b4070538f0 0x55b407055d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a28958ef4451d79 server_cookie=50c871ef6c3fd731 in_seq=0 out_seq=0 2024-08-16T19:11:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.070+0000 7eff49ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.081+0000 7eff49ffb640 1 -- 172.21.15.62:0/3969741464 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b406f3a4f0 con 0x55b4070538f0 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.081+0000 7eff49ffb640 10 client.15549 handle_command_reply: tid=0 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.081+0000 7eff6e948740 1 client.15549 shutdown 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 msgr2=0x55b406fd2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7eff5801ffd0 tx=0x7eff58004600 comp rx=0 tx=0).stop 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 shutdown_connections 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b4070538f0 0x55b407055d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff49ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b406fd8310 0x55b406fd2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b406fcda50 0x55b406fd2870 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 --2- 172.21.15.62:0/3969741464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b406fcd130 0x55b406fd2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 >> 172.21.15.62:0/3969741464 conn(0x55b406fc96c0 msgr2=0x55b406fc7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 shutdown_connections 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 1 -- 172.21.15.62:0/3969741464 wait complete. 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.082+0000 7eff6e948740 20 client.15549 trim_cache size 0 max 16384 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 28, 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 2, 2024-08-16T19:11:10.084 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "d21bfdec": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 2, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 143.6, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:11:10.085 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:11:10.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 msgr2=0x7eff4c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 0x7eff4c0a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff640feba0 tx=0x7eff58005f90 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 msgr2=0x7eff64198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005e30 tx=0x7eff5c032fb0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 shutdown_connections 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7eff4c0a2e90 0x7eff4c0a5350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff64107fd0 0x7eff6419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff64103410 0x7eff64198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 --2- 172.21.15.62:0/1426141221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff64102a60 0x7eff64198310 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 >> 172.21.15.62:0/1426141221 conn(0x7eff640fc500 msgr2=0x7eff64105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 shutdown_connections 2024-08-16T19:11:10.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:10.083+0000 7eff6c54a640 1 -- 172.21.15.62:0/1426141221 wait complete. 2024-08-16T19:11:10.781 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd21bfdec', quiesce took 4.9 seconds. Will release after: 18.700000000000003 2024-08-16T19:11:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:11 smithi062 ceph-mon[32993]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-08-16T19:11:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:11 smithi155 ceph-mon[37413]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-08-16T19:11:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:11 smithi161 ceph-mon[37576]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-08-16T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:13 smithi062 ceph-mon[32993]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-16T19:11:13.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:13 smithi155 ceph-mon[37413]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-16T19:11:13.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:13 smithi161 ceph-mon[37576]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-08-16T19:11:15.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:15 smithi062 ceph-mon[32993]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-16T19:11:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:15 smithi155 ceph-mon[37413]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-16T19:11:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:15 smithi161 ceph-mon[37576]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-08-16T19:11:17.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:17 smithi062 ceph-mon[32993]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-08-16T19:11:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:17 smithi155 ceph-mon[37413]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-08-16T19:11:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:17 smithi161 ceph-mon[37576]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-08-16T19:11:19.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:19 smithi062 ceph-mon[32993]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-16T19:11:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:19 smithi155 ceph-mon[37413]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-16T19:11:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:19 smithi161 ceph-mon[37576]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-16T19:11:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:21 smithi062 ceph-mon[32993]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-08-16T19:11:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:21 smithi155 ceph-mon[37413]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-08-16T19:11:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:21 smithi161 ceph-mon[37576]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-08-16T19:11:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:23 smithi062 ceph-mon[32993]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-08-16T19:11:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:23 smithi155 ceph-mon[37413]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-08-16T19:11:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:23 smithi161 ceph-mon[37576]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-08-16T19:11:25.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:25 smithi062 ceph-mon[32993]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-16T19:11:25.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:25 smithi155 ceph-mon[37413]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-16T19:11:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:25 smithi161 ceph-mon[37576]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-08-16T19:11:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:27 smithi062 ceph-mon[32993]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-16T19:11:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:27 smithi155 ceph-mon[37413]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-16T19:11:27.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:27 smithi161 ceph-mon[37576]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-08-16T19:11:29.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:11:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:29 smithi062 ceph-mon[32993]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-08-16T19:11:29.779 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:11:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:29 smithi155 ceph-mon[37413]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-08-16T19:11:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:29 smithi161 ceph-mon[37576]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-08-16T19:11:30.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.310+0000 7f85f3343640 1 -- 172.21.15.62:0/2682116038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec103400 msgr2=0x7f85ec10b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.310+0000 7f85f3343640 1 --2- 172.21.15.62:0/2682116038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec103400 0x7f85ec10b8f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009860 tx=0x7f85dc02a3d0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 -- 172.21.15.62:0/2682116038 shutdown_connections 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 --2- 172.21.15.62:0/2682116038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec10bec0 0x7f85ec10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 --2- 172.21.15.62:0/2682116038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec103400 0x7f85ec10b8f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 --2- 172.21.15.62:0/2682116038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec102ae0 0x7f85ec102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 -- 172.21.15.62:0/2682116038 >> 172.21.15.62:0/2682116038 conn(0x7f85ec0fc500 msgr2=0x7f85ec0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 -- 172.21.15.62:0/2682116038 shutdown_connections 2024-08-16T19:11:30.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.311+0000 7f85f3343640 1 -- 172.21.15.62:0/2682116038 wait complete. 2024-08-16T19:11:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.312+0000 7f85f3343640 1 Processor -- start 2024-08-16T19:11:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.313+0000 7f85f3343640 1 -- start start 2024-08-16T19:11:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.313+0000 7f85f3343640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 0x7f85ec197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.313+0000 7f85f3343640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:30.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.313+0000 7f85f3343640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec10bec0 0x7f85ec19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f08de640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f08de640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37232/0 (socket says 172.21.15.62:37232) 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f10df640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 0x7f85ec197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f10df640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 0x7f85ec197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:41104/0 (socket says 172.21.15.62:41104) 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f18e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec10bec0 0x7f85ec19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:30.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f08de640 1 -- 172.21.15.62:0/4279127732 learned_addr learned my addr 172.21.15.62:0/4279127732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec111ca0 con 0x7f85ec103400 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec111b20 con 0x7f85ec102ae0 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.314+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec111e20 con 0x7f85ec10bec0 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 -- 172.21.15.62:0/4279127732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec10bec0 msgr2=0x7f85ec19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec10bec0 0x7f85ec19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 -- 172.21.15.62:0/4279127732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 msgr2=0x7f85ec197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 0x7f85ec197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec19cf50 con 0x7f85ec103400 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f08de640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0309c0 tx=0x7f85dc0309f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc040070 con 0x7f85ec103400 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec19d1e0 con 0x7f85ec103400 2024-08-16T19:11:30.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.315+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a4b20 con 0x7f85ec103400 2024-08-16T19:11:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.316+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85dc004460 con 0x7f85ec103400 2024-08-16T19:11:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.316+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc04a060 con 0x7f85ec103400 2024-08-16T19:11:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.318+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec199300 con 0x7f85ec103400 2024-08-16T19:11:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.319+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f85dc043070 con 0x7f85ec103400 2024-08-16T19:11:30.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.320+0000 7f85e27fc640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 0x7f85d80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.320+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f85dc0f03b0 con 0x7f85ec103400 2024-08-16T19:11:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.321+0000 7f85f10df640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 0x7f85d80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:30.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.322+0000 7f85f10df640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 0x7f85d80a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009bd0 tx=0x7f85e8005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:30.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.326+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f85dc0bc470 con 0x7f85ec103400 2024-08-16T19:11:30.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.755+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85ec1a4ec0 con 0x7f85ec103400 2024-08-16T19:11:30.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.756+0000 7f85e27fc640 1 -- 172.21.15.62:0/4279127732 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f85dc0c0770 con 0x7f85ec103400 2024-08-16T19:11:30.759 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:11:30.760 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"btime":"2024-08-16T19:10:26:918579+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:10:26.868424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:11:30.760 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 msgr2=0x7f85d80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 0x7f85d80a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009bd0 tx=0x7f85e8005f90 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 msgr2=0x7f85ec1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0309c0 tx=0x7f85dc0309f0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 shutdown_connections 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f85d80a2e90 0x7f85d80a5350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85ec10bec0 0x7f85ec19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85ec103400 0x7f85ec1984e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.759+0000 7f85f3343640 1 --2- 172.21.15.62:0/4279127732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85ec102ae0 0x7f85ec197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:30.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.760+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 >> 172.21.15.62:0/4279127732 conn(0x7f85ec0fc500 msgr2=0x7f85ec0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.760+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 shutdown_connections 2024-08-16T19:11:30.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:30.760+0000 7f85f3343640 1 -- 172.21.15.62:0/4279127732 wait complete. 2024-08-16T19:11:31.549 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d21bfdec --release --await' 2024-08-16T19:11:31.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d21bfdec --release --await 2024-08-16T19:11:31.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:31 smithi062 ceph-mon[32993]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-08-16T19:11:31.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:31 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4279127732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:31 smithi155 ceph-mon[37413]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-08-16T19:11:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:31 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4279127732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:31 smithi161 ceph-mon[37576]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-08-16T19:11:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:31 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4279127732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:11:31.896 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:11:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 -- 172.21.15.62:0/2328185902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0100b50 msgr2=0x7f1dd0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/2328185902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0100b50 0x7f1dd0100f30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009900 tx=0x7f1dc002a480 comp rx=0 tx=0).stop 2024-08-16T19:11:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 -- 172.21.15.62:0/2328185902 shutdown_connections 2024-08-16T19:11:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/2328185902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0102c20 0x7f1dd010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/2328185902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 0x7f1dd01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/2328185902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0100b50 0x7f1dd0100f30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 -- 172.21.15.62:0/2328185902 >> 172.21.15.62:0/2328185902 conn(0x7f1dd00fc500 msgr2=0x7f1dd00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 -- 172.21.15.62:0/2328185902 shutdown_connections 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.411+0000 7f1dd49a6640 1 -- 172.21.15.62:0/2328185902 wait complete. 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.412+0000 7f1dd49a6640 1 Processor -- start 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.412+0000 7f1dd49a6640 1 -- start start 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0100b50 0x7f1dd0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 0x7f1dd0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0110cb0 con 0x7f1dd0102c20 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0110b30 con 0x7f1dd0100b50 2024-08-16T19:11:32.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dd49a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0110e30 con 0x7f1dd01022a0 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dce59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0100b50 0x7f1dd0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 0x7f1dd0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36526/0 (socket says 172.21.15.62:36526) 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 -- 172.21.15.62:0/3137749950 learned_addr learned my addr 172.21.15.62:0/3137749950 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 -- 172.21.15.62:0/3137749950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 msgr2=0x7f1dd0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 0x7f1dd0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 -- 172.21.15.62:0/3137749950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0100b50 msgr2=0x7f1dd0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0100b50 0x7f1dd0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.413+0000 7f1dced9d640 1 -- 172.21.15.62:0/3137749950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd019d200 con 0x7f1dd0102c20 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dced9d640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dbf7fe640 1 -- 172.21.15.62:0/3137749950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8017070 con 0x7f1dd0102c20 2024-08-16T19:11:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dbf7fe640 1 -- 172.21.15.62:0/3137749950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc8003e50 con 0x7f1dd0102c20 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd019d4f0 con 0x7f1dd0102c20 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dbf7fe640 1 -- 172.21.15.62:0/3137749950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80106c0 con 0x7f1dd0102c20 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.414+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01033b0 con 0x7f1dd0102c20 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.417+0000 7f1dd6da4740 1 Processor -- start 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.417+0000 7f1dbf7fe640 1 -- 172.21.15.62:0/3137749950 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f1dc8003970 con 0x7f1dd0102c20 2024-08-16T19:11:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 -- start start 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd537d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd542b70 0x5561dd544f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5561dd49dc60 con 0x5561dd537990 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5561dd435b90 con 0x5561dd5382b0 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5561dd499c20 con 0x5561dd542b70 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dce59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dce59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:48306/0 (socket says 172.21.15.62:48306) 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.418+0000 7f1dce59c640 1 -- 172.21.15.62:0/195875027 learned_addr learned my addr 172.21.15.62:0/195875027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dbf7fe640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 0x7f1db80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dce59c640 1 -- 172.21.15.62:0/195875027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd542b70 msgr2=0x5561dd544f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dce59c640 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd542b70 0x5561dd544f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dce59c640 1 -- 172.21.15.62:0/195875027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 msgr2=0x5561dd537d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd537d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dce59c640 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd537d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dce59c640 1 -- 172.21.15.62:0/195875027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dd1ad580 con 0x5561dd5382b0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dbf7fe640 1 -- 172.21.15.62:0/3137749950 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1dc80cf790 con 0x7f1dd0102c20 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd537d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.419+0000 7f1dced9d640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 0x7f1db80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.420+0000 7f1dced9d640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 0x7f1db80a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80cca30 tx=0x7f1dc80cfa90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.420+0000 7f1dce59c640 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dc002a9a0 tx=0x7f1dc0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9e6aa3cc5e8674 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.420+0000 7f1dbcff9640 1 -- 172.21.15.62:0/195875027 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035030 con 0x5561dd5382b0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.420+0000 7f1dbcff9640 1 -- 172.21.15.62:0/195875027 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc0040070 con 0x5561dd5382b0 2024-08-16T19:11:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.420+0000 7f1dbcff9640 1 -- 172.21.15.62:0/195875027 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0031930 con 0x5561dd5382b0 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 -- 172.21.15.62:0/195875027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 msgr2=0x5561dd542630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dc002a9a0 tx=0x7f1dc0005e80 comp rx=0 tx=0).stop 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 -- 172.21.15.62:0/195875027 shutdown_connections 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd542b70 0x5561dd544f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd542630 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/195875027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd537d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 -- 172.21.15.62:0/195875027 >> 172.21.15.62:0/195875027 conn(0x5561dd533f20 msgr2=0x5561dd534300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 -- 172.21.15.62:0/195875027 shutdown_connections 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.421+0000 7f1dd6da4740 1 -- 172.21.15.62:0/195875027 wait complete. 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 -- start start 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 Processor -- start 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd53fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd53ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5561dd5457e0 con 0x5561dd542b70 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5561dd43b7c0 con 0x5561dd537990 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.422+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5561dd545960 con 0x5561dd5382b0 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36544/0 (socket says 172.21.15.62:36544) 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2856674259 learned_addr learned my addr 172.21.15.62:0/2856674259 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd53ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd53fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dd6da4740 10 client.0 _ll_register_callbacks cb 0x5561dd444f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2856674259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 msgr2=0x5561dd53ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd53ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 msgr2=0x5561dd53fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd53fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2856674259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dd277960 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd53ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009710 tx=0x7f1dc4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a99996589e70dd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.423+0000 7f1db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4004260 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1dc4004400 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc0040070 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1dc401d8d0 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1dd6da4740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561dd5517b0 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc401f080 con 0x5561dd542b70 2024-08-16T19:11:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.424+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1dc400f7c0 con 0x5561dd542b70 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 10 client.15564 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 20 client.15564 populate_metadata read hostname 'smithi062' 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd552c70 0x5561dd555060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 4 client.15564 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1dc0040070 con 0x5561dd552c70 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd552c70 0x5561dd555060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd552c70 0x5561dd555060 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb8d611a2f91c0cc server_cookie=672398d1077ed174 in_seq=0 out_seq=0 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.425+0000 7f1db5ffb640 10 client.15564 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.426+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2856674259 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1dc0040070 con 0x5561dd552c70 2024-08-16T19:11:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.426+0000 7f1db5ffb640 10 client.15564 handle_command_reply: tid=0 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.426+0000 7f1dd6da4740 1 client.15564 shutdown 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 msgr2=0x5561dd540510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009710 tx=0x7f1dc4005850 comp rx=0 tx=0).stop 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 shutdown_connections 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd552c70 0x5561dd555060 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd542b70 0x5561dd540510 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd53ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2856674259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd53fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 >> 172.21.15.62:0/2856674259 conn(0x5561dd533f20 msgr2=0x5561dd539c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1db5ffb640 0 client.15564 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 shutdown_connections 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2856674259 wait complete. 2024-08-16T19:11:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.427+0000 7f1dd6da4740 20 client.15564 trim_cache size 0 max 16384 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 Processor -- start 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 -- start start 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd540360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd55a8c0 0x5561dd55ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01acaf0 con 0x5561dd537990 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01ae630 con 0x5561dd5382b0 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01ad1c0 con 0x5561dd55a8c0 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36550/0 (socket says 172.21.15.62:36550) 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/739423429 learned_addr learned my addr 172.21.15.62:0/739423429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dce59c640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd540360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.461+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd55a8c0 0x5561dd55ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/739423429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd55a8c0 msgr2=0x5561dd55ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd55a8c0 0x5561dd55ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/739423429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 msgr2=0x5561dd540360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd540360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/739423429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561dd555690 con 0x5561dd537990 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dce59c640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd540360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40099c0 tx=0x7f1dc4009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ea938c1350fd4e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1db7fff640 1 -- 172.21.15.62:0/739423429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400f790 con 0x5561dd537990 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1db7fff640 1 -- 172.21.15.62:0/739423429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc400f930 con 0x5561dd537990 2024-08-16T19:11:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.462+0000 7f1db7fff640 1 -- 172.21.15.62:0/739423429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401f330 con 0x5561dd537990 2024-08-16T19:11:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- 172.21.15.62:0/739423429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 msgr2=0x5561dd533080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40099c0 tx=0x7f1dc4009950 comp rx=0 tx=0).stop 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- 172.21.15.62:0/739423429 shutdown_connections 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd55a8c0 0x5561dd55ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd5382b0 0x5561dd540360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/739423429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd537990 0x5561dd533080 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- 172.21.15.62:0/739423429 >> 172.21.15.62:0/739423429 conn(0x5561dd533f20 msgr2=0x5561dd53ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- 172.21.15.62:0/739423429 shutdown_connections 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- 172.21.15.62:0/739423429 wait complete. 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 -- start start 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.463+0000 7f1dd6da4740 1 Processor -- start 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd546f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd55a8c0 0x5561dd547450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01adf60 con 0x5561dd55a8c0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01ad890 con 0x5561dd537990 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd01aaad0 con 0x5561dd5382b0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dd6da4740 10 client.0 _ll_register_callbacks cb 0x7f1dc401fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:48324/0 (socket says 172.21.15.62:48324) 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2236143175 learned_addr learned my addr 172.21.15.62:0/2236143175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd546f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd55a8c0 0x5561dd547450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2236143175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 msgr2=0x5561dd546f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd546f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2236143175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd55a8c0 msgr2=0x5561dd547450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:11:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd55a8c0 0x5561dd547450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.464+0000 7f1dcdd9b640 1 -- 172.21.15.62:0/2236143175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561dd535090 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1dcdd9b640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401dd80 tx=0x7f1dc401fdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1302813746d915b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1db5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400f790 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc400f930 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1dc401f0d0 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1dc401f720 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1dd6da4740 10 client.25193 fetch_fsmap learned FSMap version 28 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5561dd533d50 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.465+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc400f930 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1dc4003ae0 con 0x5561dd537990 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 10 client.25193 fetch_fsmap finished waiting for FSMap version 28 2024-08-16T19:11:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 10 client.25193 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 20 client.25193 populate_metadata read hostname 'smithi062' 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd5bdec0 0x5561dd5c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 4 client.25193 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d21bfdec", "release": true, "await": true}] 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d21bfdec", "release": true, "await": true}) -- 0x5561dd4ab1d0 con 0x5561dd5bdec0 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.466+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd5bdec0 0x5561dd5c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.467+0000 7f1db5ffb640 10 client.25193 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:32.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.467+0000 7f1dce59c640 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd5bdec0 0x5561dd5c02b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0f4719ebcf9c46 server_cookie=f4f9d80aa9c99ff4 in_seq=0 out_seq=0 2024-08-16T19:11:32.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1db5ffb640 1 -- 172.21.15.62:0/2236143175 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5561dd4ab1d0 con 0x5561dd5bdec0 2024-08-16T19:11:32.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1db5ffb640 10 client.25193 handle_command_reply: tid=0 2024-08-16T19:11:32.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1dd6da4740 1 client.25193 shutdown 2024-08-16T19:11:32.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 msgr2=0x5561dd5469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc401dd80 tx=0x7f1dc401fdb0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 shutdown_connections 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.469+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5561dd5bdec0 0x5561dd5c02b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1db5ffb640 0 client.25193 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5561dd55a8c0 0x5561dd547450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5561dd5382b0 0x5561dd546f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 --2- 172.21.15.62:0/2236143175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5561dd537990 0x5561dd5469d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 >> 172.21.15.62:0/2236143175 conn(0x5561dd533f20 msgr2=0x5561dd5393e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 shutdown_connections 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 1 -- 172.21.15.62:0/2236143175 wait complete. 2024-08-16T19:11:32.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd6da4740 20 client.25193 trim_cache size 0 max 16384 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 28, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 4, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "d21bfdec": { 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 4, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:11:32.474 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 143.6, 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:11:32.475 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 msgr2=0x7f1db80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 0x7f1db80a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80cca30 tx=0x7f1dc80cfa90 comp rx=0 tx=0).stop 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 msgr2=0x7f1dd019cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.470+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc800b260 comp rx=0 tx=0).stop 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 shutdown_connections 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1db80a2e10 0x7f1db80a52d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1dd0102c20 0x7f1dd019cb20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1dd01022a0 0x7f1dd0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 --2- 172.21.15.62:0/3137749950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1dd0100b50 0x7f1dd0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:11:32.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 >> 172.21.15.62:0/3137749950 conn(0x7f1dd00fc500 msgr2=0x7f1dd0069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:11:32.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 shutdown_connections 2024-08-16T19:11:32.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:11:32.471+0000 7f1dd49a6640 1 -- 172.21.15.62:0/3137749950 wait complete. 2024-08-16T19:11:33.190 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd21bfdec', total seconds elapsed: 27.3 2024-08-16T19:11:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:33 smithi062 ceph-mon[32993]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:11:33.480 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:11:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:11:33.481 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:11:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:33 smithi155 ceph-mon[37413]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:11:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:11:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:11:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:11:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:33 smithi161 ceph-mon[37576]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:11:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:11:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:11:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:11:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:35 smithi062 ceph-mon[32993]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:35 smithi155 ceph-mon[37413]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:35 smithi161 ceph-mon[37576]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:37 smithi062 ceph-mon[32993]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:37 smithi155 ceph-mon[37413]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:37.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:37 smithi161 ceph-mon[37576]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:39.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:39 smithi062 ceph-mon[32993]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:39.776 INFO:tasks.workunit.client.0.smithi062.stderr:Updating files: 93% (12046/12849) Updating files: 94% (12079/12849) Updating files: 95% (12207/12849) Updating files: 96% (12336/12849) Updating files: 97% (12464/12849) Updating files: 98% (12593/12849) Updating files: 99% (12721/12849) Updating files: 100% (12849/12849) Updating files: 100% (12849/12849), done. 2024-08-16T19:11:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:39 smithi155 ceph-mon[37413]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:39.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:39 smithi161 ceph-mon[37576]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:40.640 INFO:tasks.workunit.client.0.smithi062.stderr:Note: switching to '998c2f980771f4723d937d5cd7891f92dd116699'. 2024-08-16T19:11:40.640 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.640 INFO:tasks.workunit.client.0.smithi062.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-16T19:11:40.640 INFO:tasks.workunit.client.0.smithi062.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-16T19:11:40.640 INFO:tasks.workunit.client.0.smithi062.stderr:state without impacting any branches by switching back to a branch. 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: git switch -c 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr:Or undo this operation with: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: git switch - 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr: 2024-08-16T19:11:40.641 INFO:tasks.workunit.client.0.smithi062.stderr:HEAD is now at 998c2f98077 Merge pull request #59196 from guits/cv-backport-tpm2 2024-08-16T19:11:40.645 DEBUG:teuthology.orchestra.run.smithi062:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-16T19:11:40.776 INFO:tasks.workunit.client.0.smithi062.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-16T19:11:40.778 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-16T19:11:40.778 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-16T19:11:41.504 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-16T19:11:41.570 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-16T19:11:41.610 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-16T19:11:41.611 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-16T19:11:41.612 INFO:tasks.workunit.client.0.smithi062.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-16T19:11:41.652 INFO:tasks.workunit.client.0.smithi062.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-16T19:11:41.656 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:11:41.656 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-16T19:11:41.672 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-08-16T19:11:41.673 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-08-16T19:11:41.674 DEBUG:teuthology.orchestra.run.smithi062:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=998c2f980771f4723d937d5cd7891f92dd116699 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-08-16T19:11:41.736 INFO:tasks.workunit.client.0.smithi062.stderr:+ dbench 1 2024-08-16T19:11:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:41 smithi062 ceph-mon[32993]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:41.822 INFO:tasks.workunit.client.0.smithi062.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-16T19:11:41.822 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-08-16T19:11:41.822 INFO:tasks.workunit.client.0.smithi062.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-16T19:11:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:41 smithi155 ceph-mon[37413]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:41.835 INFO:tasks.workunit.client.0.smithi062.stdout:0 of 1 processes prepared for launch 0 sec 2024-08-16T19:11:41.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:41 smithi161 ceph-mon[37576]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-08-16T19:11:41.935 INFO:tasks.workunit.client.0.smithi062.stdout:1 of 1 processes prepared for launch 0 sec 2024-08-16T19:11:41.935 INFO:tasks.workunit.client.0.smithi062.stdout:releasing clients 2024-08-16T19:11:42.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 2975 33.51 MB/sec warmup 1 sec latency 10.425 ms 2024-08-16T19:11:43.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:43 smithi062 ceph-mon[32993]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:43.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:43 smithi155 ceph-mon[37413]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:43.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:43 smithi161 ceph-mon[37576]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-08-16T19:11:43.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 6009 23.62 MB/sec warmup 2 sec latency 12.782 ms 2024-08-16T19:11:44.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 8625 21.44 MB/sec warmup 3 sec latency 11.219 ms 2024-08-16T19:11:45.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:45 smithi062 ceph-mon[32993]: pgmap v41: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 352 KiB/s wr, 53 op/s 2024-08-16T19:11:45.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:45 smithi155 ceph-mon[37413]: pgmap v41: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 352 KiB/s wr, 53 op/s 2024-08-16T19:11:45.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:45 smithi161 ceph-mon[37576]: pgmap v41: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 352 KiB/s wr, 53 op/s 2024-08-16T19:11:45.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 11454 19.71 MB/sec warmup 4 sec latency 66.731 ms 2024-08-16T19:11:46.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 14645 19.49 MB/sec warmup 5 sec latency 14.329 ms 2024-08-16T19:11:47.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:47 smithi062 ceph-mon[32993]: pgmap v42: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:47 smithi155 ceph-mon[37413]: pgmap v42: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:47.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:47 smithi161 ceph-mon[37576]: pgmap v42: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:47.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 18002 19.37 MB/sec warmup 6 sec latency 14.422 ms 2024-08-16T19:11:48.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 21308 19.15 MB/sec warmup 7 sec latency 10.048 ms 2024-08-16T19:11:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:49 smithi155 ceph-mon[37413]: pgmap v43: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:49.876 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:49 smithi062 ceph-mon[32993]: pgmap v43: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:49.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:49 smithi161 ceph-mon[37576]: pgmap v43: 129 pgs: 129 active+clean; 5.0 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 352 KiB/s wr, 49 op/s 2024-08-16T19:11:49.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 23815 17.94 MB/sec warmup 8 sec latency 15.094 ms 2024-08-16T19:11:50.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 26736 17.81 MB/sec warmup 9 sec latency 9.687 ms 2024-08-16T19:11:51.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:51 smithi155 ceph-mon[37413]: pgmap v44: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-08-16T19:11:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:51 smithi161 ceph-mon[37576]: pgmap v44: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-08-16T19:11:51.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 29329 17.47 MB/sec warmup 10 sec latency 36.330 ms 2024-08-16T19:11:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:51 smithi062 ceph-mon[32993]: pgmap v44: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-08-16T19:11:52.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 32416 17.55 MB/sec warmup 11 sec latency 45.914 ms 2024-08-16T19:11:53.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:53 smithi155 ceph-mon[37413]: pgmap v45: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:11:53.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:53 smithi161 ceph-mon[37576]: pgmap v45: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:11:53.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 35736 17.66 MB/sec warmup 12 sec latency 14.735 ms 2024-08-16T19:11:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:53 smithi062 ceph-mon[32993]: pgmap v45: 129 pgs: 129 active+clean; 12 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:11:54.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 38266 17.17 MB/sec warmup 13 sec latency 19.560 ms 2024-08-16T19:11:55.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:55 smithi155 ceph-mon[37413]: pgmap v46: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 221 op/s 2024-08-16T19:11:55.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:55 smithi161 ceph-mon[37576]: pgmap v46: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 221 op/s 2024-08-16T19:11:55.935 INFO:tasks.workunit.client.0.smithi062.stdout: 1 40867 16.91 MB/sec warmup 14 sec latency 10.213 ms 2024-08-16T19:11:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:55 smithi062 ceph-mon[32993]: pgmap v46: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 1.8 MiB/s wr, 221 op/s 2024-08-16T19:11:56.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:56 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:11:56.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:56 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:11:56.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42582 16.58 MB/sec warmup 15 sec latency 588.397 ms 2024-08-16T19:11:56.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:56 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:11:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:57 smithi155 ceph-mon[37413]: pgmap v47: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:11:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:57 smithi161 ceph-mon[37576]: pgmap v47: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:11:57.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42582 15.55 MB/sec warmup 16 sec latency 1588.545 ms 2024-08-16T19:11:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:57 smithi062 ceph-mon[32993]: pgmap v47: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:11:58.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42607 14.63 MB/sec warmup 17 sec latency 2210.367 ms 2024-08-16T19:11:59.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:11:59 smithi155 ceph-mon[37413]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:11:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:11:59 smithi161 ceph-mon[37576]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:11:59.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42607 13.82 MB/sec warmup 18 sec latency 1361.640 ms 2024-08-16T19:11:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:11:59 smithi062 ceph-mon[32993]: pgmap v48: 129 pgs: 129 active+clean; 19 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-08-16T19:12:00.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42834 13.17 MB/sec warmup 19 sec latency 2005.056 ms 2024-08-16T19:12:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:01 smithi155 ceph-mon[37413]: pgmap v49: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-08-16T19:12:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:01 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:12:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:01 smithi155 ceph-mon[37413]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-08-16T19:12:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:01 smithi161 ceph-mon[37576]: pgmap v49: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-08-16T19:12:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:01 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:12:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:01 smithi161 ceph-mon[37576]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-08-16T19:12:01.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42974 12.52 MB/sec warmup 20 sec latency 844.591 ms 2024-08-16T19:12:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:01 smithi062 ceph-mon[32993]: pgmap v49: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-08-16T19:12:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:01 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:12:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:01 smithi062 ceph-mon[32993]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-08-16T19:12:02.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42986 11.93 MB/sec warmup 21 sec latency 1793.374 ms 2024-08-16T19:12:03.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:03 smithi155 ceph-mon[37413]: pgmap v50: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:12:03.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:03 smithi161 ceph-mon[37576]: pgmap v50: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:12:03.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42986 11.38 MB/sec warmup 22 sec latency 1038.218 ms 2024-08-16T19:12:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:03 smithi062 ceph-mon[32993]: pgmap v50: 129 pgs: 129 active+clean; 21 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-08-16T19:12:04.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 42986 10.89 MB/sec warmup 23 sec latency 2038.298 ms 2024-08-16T19:12:05.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:05 smithi155 ceph-mon[37413]: pgmap v51: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.6 MiB/s wr, 267 op/s 2024-08-16T19:12:05.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:05 smithi161 ceph-mon[37576]: pgmap v51: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.6 MiB/s wr, 267 op/s 2024-08-16T19:12:05.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 43349 10.45 MB/sec warmup 24 sec latency 2685.306 ms 2024-08-16T19:12:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:05 smithi062 ceph-mon[32993]: pgmap v51: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.6 MiB/s wr, 267 op/s 2024-08-16T19:12:06.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 43387 10.03 MB/sec warmup 25 sec latency 988.479 ms 2024-08-16T19:12:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:07 smithi155 ceph-mon[37413]: pgmap v52: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:07 smithi161 ceph-mon[37576]: pgmap v52: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:07.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 43387 9.65 MB/sec warmup 26 sec latency 1988.595 ms 2024-08-16T19:12:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:07 smithi062 ceph-mon[32993]: pgmap v52: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:08.936 INFO:tasks.workunit.client.0.smithi062.stdout: 1 43387 9.29 MB/sec warmup 27 sec latency 2988.692 ms 2024-08-16T19:12:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:09 smithi155 ceph-mon[37413]: pgmap v53: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:09.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:09 smithi161 ceph-mon[37576]: pgmap v53: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:09.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 43387 8.96 MB/sec warmup 28 sec latency 3988.751 ms 2024-08-16T19:12:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:09 smithi062 ceph-mon[32993]: pgmap v53: 129 pgs: 129 active+clean; 39 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 4.9 MiB/s wr, 179 op/s 2024-08-16T19:12:10.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 45065 8.90 MB/sec warmup 29 sec latency 4364.812 ms 2024-08-16T19:12:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:11 smithi155 ceph-mon[37413]: pgmap v54: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 5.0 MiB/s wr, 195 op/s 2024-08-16T19:12:11.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:11 smithi161 ceph-mon[37576]: pgmap v54: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 5.0 MiB/s wr, 195 op/s 2024-08-16T19:12:11.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 46814 8.99 MB/sec warmup 30 sec latency 36.069 ms 2024-08-16T19:12:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:11 smithi062 ceph-mon[32993]: pgmap v54: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 5.0 MiB/s wr, 195 op/s 2024-08-16T19:12:12.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 46814 8.70 MB/sec warmup 31 sec latency 1036.153 ms 2024-08-16T19:12:13.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:13 smithi155 ceph-mon[37413]: pgmap v55: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.7 MiB/s wr, 151 op/s 2024-08-16T19:12:13.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:13 smithi161 ceph-mon[37576]: pgmap v55: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.7 MiB/s wr, 151 op/s 2024-08-16T19:12:13.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 46814 8.43 MB/sec warmup 32 sec latency 2036.220 ms 2024-08-16T19:12:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:13 smithi062 ceph-mon[32993]: pgmap v55: 129 pgs: 129 active+clean; 40 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.7 MiB/s wr, 151 op/s 2024-08-16T19:12:14.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 46814 8.17 MB/sec warmup 33 sec latency 3036.283 ms 2024-08-16T19:12:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:15 smithi155 ceph-mon[37413]: pgmap v56: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-08-16T19:12:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:15 smithi161 ceph-mon[37576]: pgmap v56: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-08-16T19:12:15.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 48584 8.14 MB/sec warmup 34 sec latency 3410.072 ms 2024-08-16T19:12:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:15 smithi062 ceph-mon[32993]: pgmap v56: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-08-16T19:12:16.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50350 8.25 MB/sec warmup 35 sec latency 8.433 ms 2024-08-16T19:12:17.019 INFO:tasks.quiescer.fs.[cephfs]:Total duration (36.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:12:17.020 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.05 2024-08-16T19:12:17.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:12:17.287 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:17.670 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:17 smithi062 ceph-mon[32993]: pgmap v57: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 -- 172.21.15.62:0/2774963293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02841034b0 msgr2=0x7f02841078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 --2- 172.21.15.62:0/2774963293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02841034b0 0x7f02841078f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0278009930 tx=0x7f027802a420 comp rx=0 tx=0).stop 2024-08-16T19:12:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 -- 172.21.15.62:0/2774963293 shutdown_connections 2024-08-16T19:12:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 --2- 172.21.15.62:0/2774963293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 0x7f028410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 --2- 172.21.15.62:0/2774963293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02841034b0 0x7f02841078f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 --2- 172.21.15.62:0/2774963293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0284102b00 0x7f0284102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 -- 172.21.15.62:0/2774963293 >> 172.21.15.62:0/2774963293 conn(0x7f02840fc500 msgr2=0x7f02840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 -- 172.21.15.62:0/2774963293 shutdown_connections 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.753+0000 7f0288d27640 1 -- 172.21.15.62:0/2774963293 wait complete. 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 Processor -- start 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 -- start start 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02841034b0 0x7f028419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 0x7f028419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:17.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0284111e00 con 0x7f0284102b00 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0284111c80 con 0x7f02841034b0 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f0288d27640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0284111f80 con 0x7f0284108030 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f028259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f028259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48220/0 (socket says 172.21.15.62:48220) 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.754+0000 7f028259c640 1 -- 172.21.15.62:0/3383241844 learned_addr learned my addr 172.21.15.62:0/3383241844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f0282d9d640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 0x7f028419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f0281d9b640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02841034b0 0x7f028419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 -- 172.21.15.62:0/3383241844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 msgr2=0x7f028419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 0x7f028419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 -- 172.21.15.62:0/3383241844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02841034b0 msgr2=0x7f028419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02841034b0 0x7f028419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 -- 172.21.15.62:0/3383241844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028419f4d0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f028259c640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f027400e9f0 tx=0x7f027400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400cd80 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028419f7c0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274010910 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274018990 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.755+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a70a0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.756+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0284102ee0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.758+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0274010ac0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.758+0000 7f02737fe640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.758+0000 7f0281d9b640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.759+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f02740cfba0 con 0x7f0284102b00 2024-08-16T19:12:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.759+0000 7f0281d9b640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 0x7f026c0a51b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0278009230 tx=0x7f0278000bc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:17.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.763+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f027409c430 con 0x7f0284102b00 2024-08-16T19:12:17.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:17 smithi155 ceph-mon[37413]: pgmap v57: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:17 smithi161 ceph-mon[37576]: pgmap v57: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:17.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50355 8.02 MB/sec warmup 36 sec latency 995.424 ms 2024-08-16T19:12:17.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.991+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f028419b880 con 0x7f0284102b00 2024-08-16T19:12:17.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.991+0000 7f02737fe640 1 -- 172.21.15.62:0/3383241844 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11996 (secure 0 0 0) 0x7f027409bd30 con 0x7f0284102b00 2024-08-16T19:12:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:12:17.993 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-08-16T19:12:00:521513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:11:59.549161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 msgr2=0x7f026c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 0x7f026c0a51b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0278009230 tx=0x7f0278000bc0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 msgr2=0x7f028419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f027400e9f0 tx=0x7f027400eeb0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 shutdown_connections 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f026c0a2cf0 0x7f026c0a51b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0284108030 0x7f028419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02841034b0 0x7f028419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 --2- 172.21.15.62:0/3383241844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0284102b00 0x7f028419a520 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.993+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 >> 172.21.15.62:0/3383241844 conn(0x7f02840fc500 msgr2=0x7f0284100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.994+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 shutdown_connections 2024-08-16T19:12:17.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:17.994+0000 7f0288d27640 1 -- 172.21.15.62:0/3383241844 wait complete. 2024-08-16T19:12:18.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 138.05 --await-for 45.0' 2024-08-16T19:12:18.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 138.05 --await-for 45.0 2024-08-16T19:12:18.679 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:18 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3383241844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:18.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:18 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3383241844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:18.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:18 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3383241844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:18.920 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:18.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50355 7.81 MB/sec warmup 37 sec latency 1995.454 ms 2024-08-16T19:12:19.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 -- 172.21.15.62:0/86714463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c1078b0 msgr2=0x7f6a1c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 --2- 172.21.15.62:0/86714463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c107c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009cd0 tx=0x7f6a0c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 -- 172.21.15.62:0/86714463 shutdown_connections 2024-08-16T19:12:19.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 --2- 172.21.15.62:0/86714463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c102250 0x7f6a1c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 --2- 172.21.15.62:0/86714463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 --2- 172.21.15.62:0/86714463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c107c90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 -- 172.21.15.62:0/86714463 >> 172.21.15.62:0/86714463 conn(0x7f6a1c0fd5f0 msgr2=0x7f6a1c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 -- 172.21.15.62:0/86714463 shutdown_connections 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.355+0000 7f6a243b6640 1 -- 172.21.15.62:0/86714463 wait complete. 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 Processor -- start 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 -- start start 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c06cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c111cb0 con 0x7f6a1c102250 2024-08-16T19:12:19.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c111b30 con 0x7f6a1c1078b0 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a243b6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c111e30 con 0x7f6a1c1018b0 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48240/0 (socket says 172.21.15.62:48240) 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.356+0000 7f6a21951640 1 -- 172.21.15.62:0/1783818663 learned_addr learned my addr 172.21.15.62:0/1783818663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a22152640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a22953640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c06cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 -- 172.21.15.62:0/1783818663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 msgr2=0x7f6a1c073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c073770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 -- 172.21.15.62:0/1783818663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c1078b0 msgr2=0x7f6a1c06cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c06cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 -- 172.21.15.62:0/1783818663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1c06d3d0 con 0x7f6a1c102250 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a22152640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:12:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a21951640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000d950 tx=0x7f6a1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a1b7fe640 1 -- 172.21.15.62:0/1783818663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10017070 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a1b7fe640 1 -- 172.21.15.62:0/1783818663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a10009c30 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a1b7fe640 1 -- 172.21.15.62:0/1783818663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1000f6b0 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1c0712a0 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.357+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1c071830 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.360+0000 7f6a267b4740 1 Processor -- start 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.360+0000 7f6a267b4740 1 -- start start 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a1b7fe640 1 -- 172.21.15.62:0/1783818663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6a100109d0 con 0x7f6a1c102250 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f7185be0 0x55d3f718ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3f70ea810 con 0x55d3f71904a0 2024-08-16T19:12:19.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3f7091c70 con 0x55d3f71852c0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a267b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3f70e67d0 con 0x55d3f7185be0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a22953640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a22953640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71856a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:49276/0 (socket says 172.21.15.62:49276) 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48254/0 (socket says 172.21.15.62:48254) 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a22953640 1 -- 172.21.15.62:0/393439516 learned_addr learned my addr 172.21.15.62:0/393439516 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a22152640 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f7185be0 0x55d3f718ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 -- 172.21.15.62:0/393439516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f7185be0 msgr2=0x55d3f718ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f7185be0 0x55d3f718ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 -- 172.21.15.62:0/393439516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 msgr2=0x55d3f71856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71856a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.361+0000 7f6a21951640 1 -- 172.21.15.62:0/393439516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f6e5cba0 con 0x55d3f71904a0 2024-08-16T19:12:19.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a1b7fe640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 0x7f6a040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a21951640 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6a100cc840 tx=0x7f6a100d5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f3fc93d0c77c80 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a22953640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 0x7f6a040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a1b7fe640 1 -- 172.21.15.62:0/1783818663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f6a100cf480 con 0x7f6a1c102250 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a18dea640 1 -- 172.21.15.62:0/393439516 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100978b0 con 0x55d3f71904a0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a18dea640 1 -- 172.21.15.62:0/393439516 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a10097030 con 0x55d3f71904a0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.362+0000 7f6a18dea640 1 -- 172.21.15.62:0/393439516 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1009af70 con 0x55d3f71904a0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- 172.21.15.62:0/393439516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 msgr2=0x55d3f7192890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6a100cc840 tx=0x7f6a100d5b00 comp rx=0 tx=0).stop 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a22953640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 0x7f6a040a5230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400dd50 tx=0x7f6a1400d4a0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- 172.21.15.62:0/393439516 shutdown_connections 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f7192890 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f7185be0 0x55d3f718ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 --2- 172.21.15.62:0/393439516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- 172.21.15.62:0/393439516 >> 172.21.15.62:0/393439516 conn(0x55d3f7181850 msgr2=0x55d3f7181c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- 172.21.15.62:0/393439516 shutdown_connections 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- 172.21.15.62:0/393439516 wait complete. 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 -- start start 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.363+0000 7f6a267b4740 1 Processor -- start 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f718d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71904a0 0x55d3f718dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d3f7192fb0 con 0x55d3f7185be0 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d3f70a59a0 con 0x55d3f71852c0 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d3f7193130 con 0x55d3f71904a0 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a267b4740 10 client.0 _ll_register_callbacks cb 0x55d3f7093c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48258/0 (socket says 172.21.15.62:48258) 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 -- 172.21.15.62:0/3115977366 learned_addr learned my addr 172.21.15.62:0/3115977366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a22152640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f718d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a22953640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71904a0 0x55d3f718dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 -- 172.21.15.62:0/3115977366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71904a0 msgr2=0x55d3f718dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71904a0 0x55d3f718dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 -- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 msgr2=0x55d3f718d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f718d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 -- 172.21.15.62:0/3115977366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f70b1220 con 0x55d3f7185be0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.364+0000 7f6a21951640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6a100a3d50 tx=0x7f6a10010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aaa26c8872cf91 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10020070 con 0x55d3f7185be0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f6a100cf120 con 0x55d3f7185be0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a10097030 con 0x55d3f7185be0 2024-08-16T19:12:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a1009b150 con 0x55d3f7185be0 2024-08-16T19:12:19.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 10 client.15588 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:19.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d3f7192fb0 con 0x55d3f7185be0 2024-08-16T19:12:19.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a10097030 con 0x55d3f7185be0 2024-08-16T19:12:19.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f6a1009b870 con 0x55d3f7185be0 2024-08-16T19:12:19.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 10 client.15588 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:19.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 10 client.15588 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 20 client.15588 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f719ff50 0x55d3f71a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 4 client.15588 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.365+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f70f7f40 con 0x55d3f719ff50 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.366+0000 7f6a22152640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f719ff50 0x55d3f71a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.366+0000 7f6a01ffb640 10 client.15588 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.366+0000 7f6a22152640 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f719ff50 0x55d3f71a2340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6540801b4661ecb server_cookie=ec7d86be98403348 in_seq=0 out_seq=0 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a01ffb640 1 -- 172.21.15.62:0/3115977366 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3f70f7f40 con 0x55d3f719ff50 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a01ffb640 10 client.15588 handle_command_reply: tid=0 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 client.15588 shutdown 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 msgr2=0x55d3f718da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6a100a3d50 tx=0x7f6a10010d70 comp rx=0 tx=0).stop 2024-08-16T19:12:19.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 shutdown_connections 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f719ff50 0x55d3f71a2340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71904a0 0x55d3f718dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f7185be0 0x55d3f718da80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 --2- 172.21.15.62:0/3115977366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f718d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 >> 172.21.15.62:0/3115977366 conn(0x55d3f7181850 msgr2=0x55d3f7180b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a01ffb640 0 client.15588 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.367+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 shutdown_connections 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.368+0000 7f6a267b4740 1 -- 172.21.15.62:0/3115977366 wait complete. 2024-08-16T19:12:19.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.368+0000 7f6a267b4740 20 client.15588 trim_cache size 0 max 16384 2024-08-16T19:12:19.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 Processor -- start 2024-08-16T19:12:19.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 -- start start 2024-08-16T19:12:19.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f7180a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f718a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b7640 con 0x55d3f71904a0 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b9240 con 0x55d3f71852c0 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a267b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b7d10 con 0x55d3f71a7f70 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48268/0 (socket says 172.21.15.62:48268) 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 -- 172.21.15.62:0/691586400 learned_addr learned my addr 172.21.15.62:0/691586400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a22152640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f7180a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a22953640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f718a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 -- 172.21.15.62:0/691586400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 msgr2=0x55d3f718a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f718a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 -- 172.21.15.62:0/691586400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 msgr2=0x55d3f7180a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f7180a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.385+0000 7f6a21951640 1 -- 172.21.15.62:0/691586400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f719c210 con 0x55d3f71904a0 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a22152640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f7180a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a21951640 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009610 tx=0x7f6a10099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ecf9d622780e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a03fff640 1 -- 172.21.15.62:0/691586400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10020070 con 0x55d3f71904a0 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a03fff640 1 -- 172.21.15.62:0/691586400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a10097040 con 0x55d3f71904a0 2024-08-16T19:12:19.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a03fff640 1 -- 172.21.15.62:0/691586400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1009b3e0 con 0x55d3f71904a0 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a267b4740 1 -- 172.21.15.62:0/691586400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 msgr2=0x55d3f71a7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.386+0000 7f6a267b4740 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009610 tx=0x7f6a10099b20 comp rx=0 tx=0).stop 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- 172.21.15.62:0/691586400 shutdown_connections 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f718a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71a7a30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- 172.21.15.62:0/691586400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f7180a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- 172.21.15.62:0/691586400 >> 172.21.15.62:0/691586400 conn(0x55d3f7181850 msgr2=0x55d3f7180310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- 172.21.15.62:0/691586400 shutdown_connections 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- 172.21.15.62:0/691586400 wait complete. 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- start start 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 Processor -- start 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f7194d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b8ab0 con 0x55d3f71904a0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b83e0 con 0x55d3f71852c0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1c1b5620 con 0x55d3f71a7f70 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.387+0000 7f6a267b4740 10 client.0 _ll_register_callbacks cb 0x7f6a1400d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48270/0 (socket says 172.21.15.62:48270) 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 -- 172.21.15.62:0/41293154 learned_addr learned my addr 172.21.15.62:0/41293154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a22953640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f7194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a22152640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 -- 172.21.15.62:0/41293154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 msgr2=0x55d3f7194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f7194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 -- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 msgr2=0x55d3f71942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 -- 172.21.15.62:0/41293154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f7209c00 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a22152640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a22953640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f7194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a21951640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6a100998c0 tx=0x7f6a1009af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1542a28b3e70b649 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10020070 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f6a100a3df0 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a10097040 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a1009b3e0 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a267b4740 10 client.15594 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.388+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d3f7209c00 con 0x55d3f71904a0 2024-08-16T19:12:19.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a10097040 con 0x55d3f71904a0 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f6a10084b50 con 0x55d3f71904a0 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 10 client.15594 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 10 client.15594 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 20 client.15594 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f720b1d0 0x55d3f720d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 4 client.15594 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.05, "await_for": 45.0}] 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.05, "await_for": 45.0}) -- 0x55d3f70f2290 con 0x55d3f720b1d0 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.389+0000 7f6a22152640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f720b1d0 0x55d3f720d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.390+0000 7f6a22152640 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f720b1d0 0x55d3f720d5f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ad7d70c9592bc2 server_cookie=95c06c03c0e7aa4f in_seq=0 out_seq=0 2024-08-16T19:12:19.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.390+0000 7f6a01ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.445+0000 7f6a01ffb640 1 -- 172.21.15.62:0/41293154 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d3f70f2290 con 0x55d3f720b1d0 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.445+0000 7f6a01ffb640 10 client.15594 handle_command_reply: tid=0 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.445+0000 7f6a267b4740 1 client.15594 shutdown 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.445+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 msgr2=0x55d3f71947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.445+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6a100998c0 tx=0x7f6a1009af70 comp rx=0 tx=0).stop 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 shutdown_connections 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d3f720b1d0 0x55d3f720d5f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d3f71a7f70 0x55d3f7194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d3f71904a0 0x55d3f71947e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 --2- 172.21.15.62:0/41293154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d3f71852c0 0x55d3f71942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 >> 172.21.15.62:0/41293154 conn(0x55d3f7181850 msgr2=0x55d3f7180b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a01ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 shutdown_connections 2024-08-16T19:12:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 1 -- 172.21.15.62:0/41293154 wait complete. 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a267b4740 20 client.15594 trim_cache size 0 max 16384 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 6, 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "51019bf7": { 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 6, 2024-08-16T19:12:19.448 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 138.1, 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:19.449 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:19.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:19.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:19.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:19.450 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:12:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 msgr2=0x7f6a040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 0x7f6a040a5230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400dd50 tx=0x7f6a1400d4a0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 msgr2=0x7f6a1c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.446+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000d950 tx=0x7f6a1000de10 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 shutdown_connections 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f6a040a2d70 0x7f6a040a5230 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a1c1078b0 0x7f6a1c06cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a1c102250 0x7f6a1c06c850 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 --2- 172.21.15.62:0/1783818663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a1c1018b0 0x7f6a1c073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 >> 172.21.15.62:0/1783818663 conn(0x7f6a1c0fd5f0 msgr2=0x7f6a1c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 shutdown_connections 2024-08-16T19:12:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:19.447+0000 7f6a243b6640 1 -- 172.21.15.62:0/1783818663 wait complete. 2024-08-16T19:12:19.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:19 smithi062 ceph-mon[32993]: pgmap v58: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:19.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:19 smithi155 ceph-mon[37413]: pgmap v58: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:19 smithi161 ceph-mon[37576]: pgmap v58: 129 pgs: 129 active+clean; 42 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 61 op/s 2024-08-16T19:12:19.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 7.60 MB/sec warmup 38 sec latency 2467.054 ms 2024-08-16T19:12:20.225 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '51019bf7', quiesce took 3.2 seconds. Will release after: 14.850000000000001 2024-08-16T19:12:20.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 7.40 MB/sec warmup 39 sec latency 1528.515 ms 2024-08-16T19:12:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:21 smithi062 ceph-mon[32993]: pgmap v59: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 499 KiB/s wr, 105 op/s 2024-08-16T19:12:21.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:21 smithi155 ceph-mon[37413]: pgmap v59: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 499 KiB/s wr, 105 op/s 2024-08-16T19:12:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:21 smithi161 ceph-mon[37576]: pgmap v59: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 499 KiB/s wr, 105 op/s 2024-08-16T19:12:21.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 7.22 MB/sec warmup 40 sec latency 2528.618 ms 2024-08-16T19:12:22.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 7.04 MB/sec warmup 41 sec latency 3528.669 ms 2024-08-16T19:12:23.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:23 smithi155 ceph-mon[37413]: pgmap v60: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 418 KiB/s wr, 89 op/s 2024-08-16T19:12:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:23 smithi161 ceph-mon[37576]: pgmap v60: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 418 KiB/s wr, 89 op/s 2024-08-16T19:12:23.937 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.88 MB/sec warmup 42 sec latency 4528.782 ms 2024-08-16T19:12:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:23 smithi062 ceph-mon[32993]: pgmap v60: 129 pgs: 129 active+clean; 43 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 418 KiB/s wr, 89 op/s 2024-08-16T19:12:24.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.72 MB/sec warmup 43 sec latency 5528.879 ms 2024-08-16T19:12:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:25 smithi155 ceph-mon[37413]: pgmap v61: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 441 KiB/s wr, 99 op/s 2024-08-16T19:12:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:25 smithi161 ceph-mon[37576]: pgmap v61: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 441 KiB/s wr, 99 op/s 2024-08-16T19:12:25.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.56 MB/sec warmup 44 sec latency 6528.931 ms 2024-08-16T19:12:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:25 smithi062 ceph-mon[32993]: pgmap v61: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 441 KiB/s wr, 99 op/s 2024-08-16T19:12:26.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.42 MB/sec warmup 45 sec latency 7528.967 ms 2024-08-16T19:12:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:27 smithi155 ceph-mon[37413]: pgmap v62: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:27 smithi161 ceph-mon[37576]: pgmap v62: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:27.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.28 MB/sec warmup 46 sec latency 8529.054 ms 2024-08-16T19:12:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:27 smithi062 ceph-mon[32993]: pgmap v62: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:28.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.14 MB/sec warmup 47 sec latency 9529.139 ms 2024-08-16T19:12:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:29 smithi155 ceph-mon[37413]: pgmap v63: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:29 smithi161 ceph-mon[37576]: pgmap v63: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:29.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 6.02 MB/sec warmup 48 sec latency 10529.270 ms 2024-08-16T19:12:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:29 smithi062 ceph-mon[32993]: pgmap v63: 129 pgs: 129 active+clean; 44 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 239 KiB/s wr, 54 op/s 2024-08-16T19:12:30.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.89 MB/sec warmup 49 sec latency 11529.320 ms 2024-08-16T19:12:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:31 smithi155 ceph-mon[37413]: pgmap v64: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 804 KiB/s wr, 75 op/s 2024-08-16T19:12:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:31 smithi161 ceph-mon[37576]: pgmap v64: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 804 KiB/s wr, 75 op/s 2024-08-16T19:12:31.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.78 MB/sec warmup 50 sec latency 12529.429 ms 2024-08-16T19:12:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:31 smithi062 ceph-mon[32993]: pgmap v64: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 804 KiB/s wr, 75 op/s 2024-08-16T19:12:32.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.66 MB/sec warmup 51 sec latency 13529.510 ms 2024-08-16T19:12:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:33 smithi155 ceph-mon[37413]: pgmap v65: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 588 KiB/s wr, 31 op/s 2024-08-16T19:12:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:33 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:12:33.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:33 smithi161 ceph-mon[37576]: pgmap v65: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 588 KiB/s wr, 31 op/s 2024-08-16T19:12:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:33 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:12:33.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.55 MB/sec warmup 52 sec latency 14529.565 ms 2024-08-16T19:12:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:33 smithi062 ceph-mon[32993]: pgmap v65: 129 pgs: 129 active+clean; 50 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 588 KiB/s wr, 31 op/s 2024-08-16T19:12:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:33 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:12:34.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.45 MB/sec warmup 53 sec latency 15529.660 ms 2024-08-16T19:12:35.076 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:12:35.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:12:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:12:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:12:35.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:12:35.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:12:35.352 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 -- 172.21.15.62:0/380992371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd78102c20 msgr2=0x7fcd7810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 --2- 172.21.15.62:0/380992371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd78102c20 0x7fcd7810d770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ca0 tx=0x7fcd7002a950 comp rx=0 tx=0).stop 2024-08-16T19:12:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 -- 172.21.15.62:0/380992371 shutdown_connections 2024-08-16T19:12:35.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 --2- 172.21.15.62:0/380992371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd78102c20 0x7fcd7810d770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 --2- 172.21.15.62:0/380992371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd781022a0 0x7fcd781026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 --2- 172.21.15.62:0/380992371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 0x7fcd78100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 -- 172.21.15.62:0/380992371 >> 172.21.15.62:0/380992371 conn(0x7fcd780fc500 msgr2=0x7fcd780fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 -- 172.21.15.62:0/380992371 shutdown_connections 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.808+0000 7fcd7d500640 1 -- 172.21.15.62:0/380992371 wait complete. 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 Processor -- start 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 -- start start 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 0x7fcd78198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd78102c20 0x7fcd781a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.809+0000 7fcd7d500640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78110cb0 con 0x7fcd781022a0 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd7d500640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78110b30 con 0x7fcd78100b50 2024-08-16T19:12:35.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd7d500640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78110e30 con 0x7fcd78102c20 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35920/0 (socket says 172.21.15.62:35920) 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 -- 172.21.15.62:0/3671494178 learned_addr learned my addr 172.21.15.62:0/3671494178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd777fe640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd78102c20 0x7fcd781a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd76ffd640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 0x7fcd78198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 -- 172.21.15.62:0/3671494178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd78102c20 msgr2=0x7fcd781a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd78102c20 0x7fcd781a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 -- 172.21.15.62:0/3671494178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 msgr2=0x7fcd78198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 0x7fcd78198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 -- 172.21.15.62:0/3671494178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781a45a0 con 0x7fcd781022a0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd767fc640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00d770 tx=0x7fcd6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.810+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c017070 con 0x7fcd781022a0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.811+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781a4770 con 0x7fcd781022a0 2024-08-16T19:12:35.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.811+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6c005500 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.811+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c004e00 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.811+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a4cb0 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.812+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd78100f30 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.813+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcd6c004fa0 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.814+0000 7fcd5ffff640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 0x7fcd600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.814+0000 7fcd76ffd640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 0x7fcd600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.814+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fcd6c0cf150 con 0x7fcd781022a0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.815+0000 7fcd76ffd640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 0x7fcd600a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005d00 tx=0x7fcd6800b410 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:35.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:35.818+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcd6c09b330 con 0x7fcd781022a0 2024-08-16T19:12:35.938 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.35 MB/sec warmup 54 sec latency 16529.711 ms 2024-08-16T19:12:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:35 smithi062 ceph-mon[32993]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 608 KiB/s wr, 43 op/s 2024-08-16T19:12:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.034+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd781a4fd0 con 0x7fcd781022a0 2024-08-16T19:12:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.036+0000 7fcd5ffff640 1 -- 172.21.15.62:0/3671494178 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11996 (secure 0 0 0) 0x7fcd6c09f630 con 0x7fcd781022a0 2024-08-16T19:12:36.037 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:12:36.037 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-08-16T19:12:00:521513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:11:59.549161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:12:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 msgr2=0x7fcd600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 0x7fcd600a51b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005d00 tx=0x7fcd6800b410 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 msgr2=0x7fcd78199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00d770 tx=0x7fcd6c00dc30 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 shutdown_connections 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcd600a2cf0 0x7fcd600a51b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcd78102c20 0x7fcd781a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd781022a0 0x7fcd78199520 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 --2- 172.21.15.62:0/3671494178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd78100b50 0x7fcd78198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 >> 172.21.15.62:0/3671494178 conn(0x7fcd780fc500 msgr2=0x7fcd78106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 shutdown_connections 2024-08-16T19:12:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:36.038+0000 7fcd7d500640 1 -- 172.21.15.62:0/3671494178 wait complete. 2024-08-16T19:12:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:35 smithi161 ceph-mon[37576]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 608 KiB/s wr, 43 op/s 2024-08-16T19:12:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:35 smithi155 ceph-mon[37413]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 608 KiB/s wr, 43 op/s 2024-08-16T19:12:36.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 51019bf7 --release --await' 2024-08-16T19:12:36.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 51019bf7 --release --await 2024-08-16T19:12:36.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50356 5.25 MB/sec warmup 55 sec latency 17529.752 ms 2024-08-16T19:12:37.093 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:37.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:36 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3671494178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:37.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:36 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3671494178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:37.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:36 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3671494178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 -- 172.21.15.62:0/871008918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5734101150 msgr2=0x7f573410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 --2- 172.21.15.62:0/871008918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5734101150 0x7f573410d680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f572c009cd0 tx=0x7f572c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 -- 172.21.15.62:0/871008918 shutdown_connections 2024-08-16T19:12:37.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 --2- 172.21.15.62:0/871008918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5734101150 0x7f573410d680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 --2- 172.21.15.62:0/871008918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57341007b0 0x7f5734100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 --2- 172.21.15.62:0/871008918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 0x7f5734106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 -- 172.21.15.62:0/871008918 >> 172.21.15.62:0/871008918 conn(0x7f57340fc4d0 msgr2=0x7f57340fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 -- 172.21.15.62:0/871008918 shutdown_connections 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.589+0000 7f573bccd640 1 -- 172.21.15.62:0/871008918 wait complete. 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 Processor -- start 2024-08-16T19:12:37.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 -- start start 2024-08-16T19:12:37.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734101150 0x7f5734199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 0x7f57341a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5734110bd0 con 0x7f57341007b0 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5734110a50 con 0x7f57341067b0 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f573bccd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5734110d50 con 0x7f5734101150 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35934/0 (socket says 172.21.15.62:35934) 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.590+0000 7f5739a69640 1 -- 172.21.15.62:0/1307659999 learned_addr learned my addr 172.21.15.62:0/1307659999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739268640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734101150 0x7f5734199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f573a26a640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 0x7f57341a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 -- 172.21.15.62:0/1307659999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734101150 msgr2=0x7f5734199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734101150 0x7f5734199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 -- 172.21.15.62:0/1307659999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 msgr2=0x7f57341a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 0x7f57341a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 -- 172.21.15.62:0/1307659999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341a4600 con 0x7f57341007b0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f5739a69640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f573000d950 tx=0x7f573000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f572affd640 1 -- 172.21.15.62:0/1307659999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730017070 con 0x7f57341007b0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341a47d0 con 0x7f57341007b0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f572affd640 1 -- 172.21.15.62:0/1307659999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5730009c30 con 0x7f57341007b0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f572affd640 1 -- 172.21.15.62:0/1307659999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573000f6b0 con 0x7f57341007b0 2024-08-16T19:12:37.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.591+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a4d10 con 0x7f57341007b0 2024-08-16T19:12:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.593+0000 7f572affd640 1 -- 172.21.15.62:0/1307659999 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f57300109d0 con 0x7f57341007b0 2024-08-16T19:12:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 Processor -- start 2024-08-16T19:12:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 -- start start 2024-08-16T19:12:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c20098a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c200997370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c2008ebc60 con 0x55c20098a6c0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c200883b90 con 0x55c200989da0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f573e0cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c2008e7c20 con 0x55c200994f80 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f572affd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 0x7f57200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.594+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f573a26a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c20098a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35946/0 (socket says 172.21.15.62:35946) 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 -- 172.21.15.62:0/422766249 learned_addr learned my addr 172.21.15.62:0/422766249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739268640 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c200997370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f572affd640 1 -- 172.21.15.62:0/1307659999 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f57300cf500 con 0x7f57341007b0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f573a26a640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 0x7f57200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 -- 172.21.15.62:0/422766249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 msgr2=0x55c200997370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c200997370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 -- 172.21.15.62:0/422766249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 msgr2=0x55c20098a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c20098a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 -- 172.21.15.62:0/422766249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2005fb580 con 0x55c20098a6c0 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739268640 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c200997370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f5739a69640 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f573009b390 tx=0x7f573006e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e0b289a453739 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f573a26a640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 0x7f57200a51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f572c009430 tx=0x7f572c02a8e0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f571ffff640 1 -- 172.21.15.62:0/422766249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730020050 con 0x55c20098a6c0 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f571ffff640 1 -- 172.21.15.62:0/422766249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5730066720 con 0x55c20098a6c0 2024-08-16T19:12:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.595+0000 7f571ffff640 1 -- 172.21.15.62:0/422766249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300a4300 con 0x55c20098a6c0 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 -- 172.21.15.62:0/422766249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 msgr2=0x55c200994a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f573009b390 tx=0x7f573006e700 comp rx=0 tx=0).stop 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 -- 172.21.15.62:0/422766249 shutdown_connections 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c200997370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c20098a6c0 0x55c200994a40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 --2- 172.21.15.62:0/422766249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c20098a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 -- 172.21.15.62:0/422766249 >> 172.21.15.62:0/422766249 conn(0x55c200986330 msgr2=0x55c200986710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 -- 172.21.15.62:0/422766249 shutdown_connections 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.596+0000 7f573e0cb740 1 -- 172.21.15.62:0/422766249 wait complete. 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 -- start start 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 Processor -- start 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c20098a6c0 0x55c200992470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c2009929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c200997af0 con 0x55c200989da0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c2008897c0 con 0x55c20098a6c0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c200997c70 con 0x55c200994f80 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573e0cb740 10 client.0 _ll_register_callbacks cb 0x55c200892f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35954/0 (socket says 172.21.15.62:35954) 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f5739a69640 1 -- 172.21.15.62:0/1121017997 learned_addr learned my addr 172.21.15.62:0/1121017997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f5739268640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c20098a6c0 0x55c200992470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.597+0000 7f573a26a640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c2009929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 -- 172.21.15.62:0/1121017997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 msgr2=0x55c2009929b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c2009929b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 -- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c20098a6c0 msgr2=0x55c200992470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c20098a6c0 0x55c200992470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 -- 172.21.15.62:0/1121017997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2006c5960 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f5739a69640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f573009fc00 tx=0x7f57300097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611181c404787a78 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300b4060 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f57300cf110 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730066720 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f573009bd90 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f573e0cb740 10 client.15612 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2009a3bf0 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5730066720 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.598+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f57300a3e70 con 0x55c200989da0 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 10 client.15612 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 20 client.15612 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2009a4f40 0x55c2009a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 4 client.15612 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2008f91d0 con 0x55c2009a4f40 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f5739268640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2009a4f40 0x55c2009a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f5739268640 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2009a4f40 0x55c2009a7330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b75d302db0a37bf server_cookie=f33450ad926be641 in_seq=0 out_seq=0 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.599+0000 7f571d7fa640 10 client.15612 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f571d7fa640 1 -- 172.21.15.62:0/1121017997 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2008f91d0 con 0x55c2009a4f40 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f571d7fa640 10 client.15612 handle_command_reply: tid=0 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 client.15612 shutdown 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 msgr2=0x55c200991f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f573009fc00 tx=0x7f57300097b0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 shutdown_connections 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c2009a4f40 0x55c2009a7330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c200994f80 0x55c2009929b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c20098a6c0 0x55c200992470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1121017997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c200991f30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f571d7fa640 0 client.15612 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 >> 172.21.15.62:0/1121017997 conn(0x55c200986330 msgr2=0x55c20098c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 shutdown_connections 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 1 -- 172.21.15.62:0/1121017997 wait complete. 2024-08-16T19:12:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.600+0000 7f573e0cb740 20 client.15612 trim_cache size 0 max 16384 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 Processor -- start 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 -- start start 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c2009ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200994f80 0x55c20098f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57341acba0 con 0x55c200989da0 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57341ae7a0 con 0x55c200994f80 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573e0cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57341ad270 con 0x55c20098a6c0 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f573a26a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200994f80 0x55c20098f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35960/0 (socket says 172.21.15.62:35960) 2024-08-16T19:12:37.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 -- 172.21.15.62:0/496372132 learned_addr learned my addr 172.21.15.62:0/496372132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:37.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739268640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c2009ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 -- 172.21.15.62:0/496372132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 msgr2=0x55c2009ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c2009ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 -- 172.21.15.62:0/496372132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200994f80 msgr2=0x55c20098f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200994f80 0x55c20098f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.618+0000 7f5739a69640 1 -- 172.21.15.62:0/496372132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c200990ee0 con 0x55c200989da0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f5739a69640 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5730010fd0 tx=0x7f5730005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5418733052047a61 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f571f7fe640 1 -- 172.21.15.62:0/496372132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730020040 con 0x55c200989da0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f571f7fe640 1 -- 172.21.15.62:0/496372132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573009fa40 con 0x55c200989da0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f571f7fe640 1 -- 172.21.15.62:0/496372132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300a3e70 con 0x55c200989da0 2024-08-16T19:12:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f573e0cb740 1 -- 172.21.15.62:0/496372132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 msgr2=0x55c2009ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.619+0000 7f573e0cb740 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5730010fd0 tx=0x7f5730005880 comp rx=0 tx=0).stop 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- 172.21.15.62:0/496372132 shutdown_connections 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200994f80 0x55c20098f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c2009ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- 172.21.15.62:0/496372132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200989da0 0x55c2009ac930 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- 172.21.15.62:0/496372132 >> 172.21.15.62:0/496372132 conn(0x55c200986330 msgr2=0x55c200992710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- 172.21.15.62:0/496372132 shutdown_connections 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- 172.21.15.62:0/496372132 wait complete. 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- start start 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 Processor -- start 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c200999420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c200999960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f57341ae010 con 0x55c200994f80 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f57341ad940 con 0x55c200989da0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.620+0000 7f573e0cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57341aab80 con 0x55c20098a6c0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573e0cb740 10 client.0 _ll_register_callbacks cb 0x7f5730097270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35976/0 (socket says 172.21.15.62:35976) 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f5739268640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c200999960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 -- 172.21.15.62:0/1360622000 learned_addr learned my addr 172.21.15.62:0/1360622000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f5739a69640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c200999420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 -- 172.21.15.62:0/1360622000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 msgr2=0x55c200999960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c200999960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 -- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 msgr2=0x55c200999420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c200999420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 -- 172.21.15.62:0/1360622000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c200a0ef40 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f5739268640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c200999960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f5739a69640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c200999420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573a26a640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f572c009cd0 tx=0x7f572c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711e42382ea38876 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f571d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c058020 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2008f36e0 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.621+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f572c004410 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f572c004610 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 10 client.15618 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c200a10240 con 0x55c200994f80 2024-08-16T19:12:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2008f36e0 con 0x55c200994f80 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f572c030070 con 0x55c200994f80 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 10 client.15618 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 20 client.15618 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c200a10ad0 0x55c200a12ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 4 client.15618 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "51019bf7", "release": true, "await": true}] 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "51019bf7", "release": true, "await": true}) -- 0x55c2008705a0 con 0x55c200a10ad0 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.622+0000 7f5739a69640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c200a10ad0 0x55c200a12ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.623+0000 7f571d7fa640 10 client.15618 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.623+0000 7f5739a69640 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c200a10ad0 0x55c200a12ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777881cc3342ed68 server_cookie=5549e575f7c91431 in_seq=0 out_seq=0 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f571d7fa640 1 -- 172.21.15.62:0/1360622000 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c2008705a0 con 0x55c200a10ad0 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f571d7fa640 10 client.15618 handle_command_reply: tid=0 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 client.15618 shutdown 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 msgr2=0x55c200a0e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f572c009cd0 tx=0x7f572c0095c0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 shutdown_connections 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c200a10ad0 0x55c200a12ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c200994f80 0x55c200a0e900 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20098a6c0 0x55c200999960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f571d7fa640 0 client.15618 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:37.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 --2- 172.21.15.62:0/1360622000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c200989da0 0x55c200999420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 >> 172.21.15.62:0/1360622000 conn(0x55c200986330 msgr2=0x55c20098b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 shutdown_connections 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 1 -- 172.21.15.62:0/1360622000 wait complete. 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.630+0000 7f573e0cb740 20 client.15618 trim_cache size 0 max 16384 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 8, 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:12:37.633 INFO:teuthology.orchestra.run.smithi062.stdout: "51019bf7": { 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 8, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 138.1, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:12:37.634 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:12:37.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 msgr2=0x7f57200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 0x7f57200a51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f572c009430 tx=0x7f572c02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 msgr2=0x7f5734199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f573000d950 tx=0x7f573000de10 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 shutdown_connections 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f57200a2cf0 0x7f57200a51b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57341067b0 0x7f57341a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5734101150 0x7f5734199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 --2- 172.21.15.62:0/1307659999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57341007b0 0x7f5734199040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 >> 172.21.15.62:0/1307659999 conn(0x7f57340fc4d0 msgr2=0x7f57340fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 shutdown_connections 2024-08-16T19:12:37.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:37.631+0000 7f573bccd640 1 -- 172.21.15.62:0/1307659999 wait complete. 2024-08-16T19:12:37.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 50858 5.19 MB/sec warmup 56 sec latency 18224.221 ms 2024-08-16T19:12:38.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:37 smithi161 ceph-mon[37576]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:37 smithi062 ceph-mon[32993]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:38.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:37 smithi155 ceph-mon[37413]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:38.461 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '51019bf7', total seconds elapsed: 21.4 2024-08-16T19:12:38.461 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.05 2024-08-16T19:12:38.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:12:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:38.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 53546 5.39 MB/sec warmup 57 sec latency 8.894 ms 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 -- 172.21.15.62:0/1647777993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 msgr2=0x7fbd8c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1647777993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c102e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009930 tx=0x7fbd7c02a420 comp rx=0 tx=0).stop 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 -- 172.21.15.62:0/1647777993 shutdown_connections 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1647777993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 0x7fbd8c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1647777993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 0x7fbd8c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1647777993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c102e40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 -- 172.21.15.62:0/1647777993 >> 172.21.15.62:0/1647777993 conn(0x7fbd8c0fc500 msgr2=0x7fbd8c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.178+0000 7fbd9148b640 1 -- 172.21.15.62:0/1647777993 shutdown_connections 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.179+0000 7fbd9148b640 1 -- 172.21.15.62:0/1647777993 wait complete. 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.179+0000 7fbd9148b640 1 Processor -- start 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.179+0000 7fbd9148b640 1 -- start start 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 0x7fbd8c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 0x7fbd8c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:39.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c111dd0 con 0x7fbd8c102a60 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c111c50 con 0x7fbd8c103410 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd9148b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c111f50 con 0x7fbd8c107fd0 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35980/0 (socket says 172.21.15.62:35980) 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 -- 172.21.15.62:0/1563745410 learned_addr learned my addr 172.21.15.62:0/1563745410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8b7fe640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 0x7fbd8c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8a7fc640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 0x7fbd8c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 -- 172.21.15.62:0/1563745410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 msgr2=0x7fbd8c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 0x7fbd8c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 -- 172.21.15.62:0/1563745410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 msgr2=0x7fbd8c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:39.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 0x7fbd8c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.180+0000 7fbd8affd640 1 -- 172.21.15.62:0/1563745410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c19d280 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd8affd640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009230 tx=0x7fbd7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c03f030 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c003ad0 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c035070 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c19d510 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.181+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c19d9a0 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.182+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c102e40 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.183+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd7c042030 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.184+0000 7fbd73fff640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 0x7fbd740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.184+0000 7fbd8a7fc640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 0x7fbd740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.184+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fbd7c0efcc0 con 0x7fbd8c102a60 2024-08-16T19:12:39.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.185+0000 7fbd8a7fc640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 0x7fbd740a5230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0feb30 tx=0x7fbd8000a2d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:39.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.189+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbd7c0bbf00 con 0x7fbd8c102a60 2024-08-16T19:12:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.412+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd8c199630 con 0x7fbd8c102a60 2024-08-16T19:12:39.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.413+0000 7fbd73fff640 1 -- 172.21.15.62:0/1563745410 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11996 (secure 0 0 0) 0x7fbd7c0c0200 con 0x7fbd8c102a60 2024-08-16T19:12:39.415 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:12:39.415 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-08-16T19:12:00:521513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:11:59.549161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:12:39.416 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 msgr2=0x7fbd740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 0x7fbd740a5230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0feb30 tx=0x7fbd8000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 msgr2=0x7fbd8c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009230 tx=0x7fbd7c004270 comp rx=0 tx=0).stop 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 shutdown_connections 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbd740a2d70 0x7fbd740a5230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd8c107fd0 0x7fbd8c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8c103410 0x7fbd8c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 --2- 172.21.15.62:0/1563745410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd8c102a60 0x7fbd8c1982d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.415+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 >> 172.21.15.62:0/1563745410 conn(0x7fbd8c0fc500 msgr2=0x7fbd8c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.416+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 shutdown_connections 2024-08-16T19:12:39.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:39.416+0000 7fbd9148b640 1 -- 172.21.15.62:0/1563745410 wait complete. 2024-08-16T19:12:39.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 55672 5.43 MB/sec warmup 58 sec latency 17.962 ms 2024-08-16T19:12:40.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 138.05 --await-for 45.0' 2024-08-16T19:12:40.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 138.05 --await-for 45.0 2024-08-16T19:12:40.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:39 smithi161 ceph-mon[37576]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:40.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:39 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1563745410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:40.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:39 smithi062 ceph-mon[32993]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:40.138 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:39 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1563745410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:40.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:39 smithi155 ceph-mon[37413]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 585 KiB/s wr, 33 op/s 2024-08-16T19:12:40.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:39 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1563745410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:40.389 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 -- 172.21.15.62:0/572449642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 msgr2=0x7fa6f4101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 --2- 172.21.15.62:0/572449642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4101d40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009930 tx=0x7fa6dc02a420 comp rx=0 tx=0).stop 2024-08-16T19:12:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 -- 172.21.15.62:0/572449642 shutdown_connections 2024-08-16T19:12:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 --2- 172.21.15.62:0/572449642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f4102280 0x7fa6f410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 --2- 172.21.15.62:0/572449642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4101d40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 --2- 172.21.15.62:0/572449642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f41078e0 0x7fa6f4107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 -- 172.21.15.62:0/572449642 >> 172.21.15.62:0/572449642 conn(0x7fa6f40fd600 msgr2=0x7fa6f40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 -- 172.21.15.62:0/572449642 shutdown_connections 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.860+0000 7fa6fa797640 1 -- 172.21.15.62:0/572449642 wait complete. 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 Processor -- start 2024-08-16T19:12:40.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 -- start start 2024-08-16T19:12:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f4102280 0x7fa6f4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f41078e0 0x7fa6f4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111ce0 con 0x7fa6f41018e0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111b60 con 0x7fa6f4102280 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6fa797640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4111e60 con 0x7fa6f41078e0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43530/0 (socket says 172.21.15.62:43530) 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3447499208 learned_addr learned my addr 172.21.15.62:0/3447499208 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f4102280 0x7fa6f4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.861+0000 7fa6f8d34640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f41078e0 0x7fa6f4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3447499208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f41078e0 msgr2=0x7fa6f4074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f41078e0 0x7fa6f4074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3447499208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f4102280 msgr2=0x7fa6f4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f4102280 0x7fa6f4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3447499208 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f41a6680 con 0x7fa6f41018e0 2024-08-16T19:12:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d950 tx=0x7fa6e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f1ffb640 1 -- 172.21.15.62:0/3447499208 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4017030 con 0x7fa6f41018e0 2024-08-16T19:12:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f1ffb640 1 -- 172.21.15.62:0/3447499208 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e4004020 con 0x7fa6f41018e0 2024-08-16T19:12:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41a6970 con 0x7fa6f41018e0 2024-08-16T19:12:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6f1ffb640 1 -- 172.21.15.62:0/3447499208 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40047a0 con 0x7fa6f41018e0 2024-08-16T19:12:40.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.862+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41a6e80 con 0x7fa6f41018e0 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.864+0000 7fa6f1ffb640 1 -- 172.21.15.62:0/3447499208 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6e40041d0 con 0x7fa6f41018e0 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 Processor -- start 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 -- start start 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e6318e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e630e6810 con 0x561e63181be0 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e6308dc70 con 0x561e631812c0 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e630e27d0 con 0x561e6318c4a0 2024-08-16T19:12:40.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43536/0 (socket says 172.21.15.62:43536) 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3306239059 learned_addr learned my addr 172.21.15.62:0/3306239059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f8d34640 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e6318e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f1ffb640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 0x7fa6e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3306239059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 msgr2=0x561e6318e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e6318e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3306239059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 msgr2=0x561e631816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.865+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3306239059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e62e58ba0 con 0x561e63181be0 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6f8d34640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 0x7fa6e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6f1ffb640 1 -- 172.21.15.62:0/3447499208 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa6e40cf0d0 con 0x7fa6f41018e0 2024-08-16T19:12:40.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc400 tx=0x7fa6e40c5df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834c3143f17b229 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6ea7fc640 1 -- 172.21.15.62:0/3306239059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e409b030 con 0x561e63181be0 2024-08-16T19:12:40.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6ea7fc640 1 -- 172.21.15.62:0/3306239059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e409b960 con 0x561e63181be0 2024-08-16T19:12:40.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.866+0000 7fa6ea7fc640 1 -- 172.21.15.62:0/3306239059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e409b260 con 0x561e63181be0 2024-08-16T19:12:40.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3306239059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 msgr2=0x561e6318bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc400 tx=0x7fa6e40c5df0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6f8d34640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 0x7fa6e00a51b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009fd0 tx=0x7fa6ec004a40 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3306239059 shutdown_connections 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e6318e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e6318bf60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3306239059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3306239059 >> 172.21.15.62:0/3306239059 conn(0x561e6317d850 msgr2=0x561e6317dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3306239059 shutdown_connections 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3306239059 wait complete. 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 -- start start 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.867+0000 7fa6fcb95740 1 Processor -- start 2024-08-16T19:12:40.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e63181be0 0x561e631899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 0x561e63189f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e6318efb0 con 0x561e6318c4a0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561e630a19a0 con 0x561e63181be0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561e6318f130 con 0x561e631812c0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f8d34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 0x561e63189f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:43258/0 (socket says 172.21.15.62:43258) 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e63181be0 0x561e631899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f8d34640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 0x561e63189f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43548/0 (socket says 172.21.15.62:43548) 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 -- 172.21.15.62:0/1923858755 learned_addr learned my addr 172.21.15.62:0/1923858755 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6fcb95740 10 client.0 _ll_register_callbacks cb 0x561e6308fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:40.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 -- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e63181be0 msgr2=0x561e631899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e63181be0 0x561e631899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 -- 172.21.15.62:0/1923858755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 msgr2=0x561e63189f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 0x561e63189f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.868+0000 7fa6f3fff640 1 -- 172.21.15.62:0/1923858755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e630ad220 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cca90 tx=0x7fa6e4009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cc2fc53577d72db server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4014070 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa6e40cc710 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e409b960 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6e409b4c0 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 10 client.25276 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e6318efb0 con 0x561e631812c0 2024-08-16T19:12:40.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e409b960 con 0x561e631812c0 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7fa6e40978d0 con 0x561e631812c0 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 10 client.25276 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 10 client.25276 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.869+0000 7fa6fcb95740 20 client.25276 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e6319be30 0x561e6319e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6fcb95740 4 client.25276 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e630f3f40 con 0x561e6319be30 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e6319be30 0x561e6319e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e6319be30 0x561e6319e220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f088e55b9503ce1 server_cookie=bb09f19ea7870202 in_seq=0 out_seq=0 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.870+0000 7fa6d37fe640 10 client.25276 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6d37fe640 1 -- 172.21.15.62:0/1923858755 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e630f3f40 con 0x561e6319be30 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6d37fe640 10 client.25276 handle_command_reply: tid=0 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 client.25276 shutdown 2024-08-16T19:12:40.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 msgr2=0x561e631894b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cca90 tx=0x7fa6e4009740 comp rx=0 tx=0).stop 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 shutdown_connections 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e6319be30 0x561e6319e220 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e6318c4a0 0x561e63189f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e63181be0 0x561e631899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1923858755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e631894b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6d37fe640 0 client.25276 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 >> 172.21.15.62:0/1923858755 conn(0x561e6317d850 msgr2=0x561e6317cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 shutdown_connections 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1923858755 wait complete. 2024-08-16T19:12:40.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.871+0000 7fa6fcb95740 20 client.25276 trim_cache size 0 max 16384 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 Processor -- start 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 -- start start 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e6318b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63188d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e6318c4a0 0x561e631a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41b0f80 con 0x561e63181be0 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41b2b80 con 0x561e6318c4a0 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.889+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41b1650 con 0x561e631812c0 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63188d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e6318b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e6318b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:43270/0 (socket says 172.21.15.62:43270) 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6f3fff640 1 -- 172.21.15.62:0/1827585867 learned_addr learned my addr 172.21.15.62:0/1827585867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6f8d34640 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e6318c4a0 0x561e631a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 -- 172.21.15.62:0/1827585867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 msgr2=0x561e6318b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e6318b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 -- 172.21.15.62:0/1827585867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e6318c4a0 msgr2=0x561e631a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e6318c4a0 0x561e631a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 -- 172.21.15.62:0/1827585867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6319e7b0 con 0x561e63181be0 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63188d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005940 tx=0x7fa6dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c140d1f58830a82f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6e9ffb640 1 -- 172.21.15.62:0/1827585867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc035040 con 0x561e63181be0 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6e9ffb640 1 -- 172.21.15.62:0/1827585867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc005020 con 0x561e63181be0 2024-08-16T19:12:40.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.890+0000 7fa6e9ffb640 1 -- 172.21.15.62:0/1827585867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc004ab0 con 0x561e63181be0 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1827585867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 msgr2=0x561e63188d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63188d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005940 tx=0x7fa6dc009260 comp rx=0 tx=0).stop 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1827585867 shutdown_connections 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e6318c4a0 0x561e631a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63188d90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/1827585867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e631812c0 0x561e6318b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1827585867 >> 172.21.15.62:0/1827585867 conn(0x561e6317d850 msgr2=0x561e63183530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1827585867 shutdown_connections 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- 172.21.15.62:0/1827585867 wait complete. 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.891+0000 7fa6fcb95740 1 -- start start 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 Processor -- start 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e63190c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41b23f0 con 0x561e63181be0 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41b1d20 con 0x561e631812c0 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41aef60 con 0x561e6318c4a0 2024-08-16T19:12:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6fcb95740 10 client.0 _ll_register_callbacks cb 0x7fa6ec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f8d34640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e63190c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:48498/0 (socket says 172.21.15.62:48498) 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6ebfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43578/0 (socket says 172.21.15.62:43578) 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3228968360 learned_addr learned my addr 172.21.15.62:0/3228968360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3228968360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 msgr2=0x561e63190c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e63190c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3228968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 msgr2=0x561e63194bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6f3fff640 1 -- 172.21.15.62:0/3228968360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e63182150 con 0x561e631812c0 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.892+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:12:40.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6f3fff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e409af60 tx=0x7fa6e40045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71bdde4509a14bdc server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4014040 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa6e40d2ac0 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6dc005020 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6e409c590 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 10 client.25256 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e6317d680 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc005020 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7fa6e40ccbe0 con 0x561e631812c0 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 10 client.25256 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 10 client.25256 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:40.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.893+0000 7fa6fcb95740 20 client.25256 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e63207890 0x561e63209cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6fcb95740 4 client.25256 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.05, "await_for": 45.0}] 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.05, "await_for": 45.0}) -- 0x561e630ee290 con 0x561e63207890 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e63207890 0x561e63209cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6d37fe640 10 client.25256 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:40.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.894+0000 7fa6ebfff640 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e63207890 0x561e63209cb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a2e3f7e3c088e2 server_cookie=a2a0312afb79eb6f in_seq=0 out_seq=0 2024-08-16T19:12:40.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.54 MB/sec warmup 59 sec latency 44.685 ms 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6d37fe640 1 -- 172.21.15.62:0/3228968360 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e630ee290 con 0x561e63207890 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6d37fe640 10 client.25256 handle_command_reply: tid=0 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 client.25256 shutdown 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 msgr2=0x561e631946b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6e409af60 tx=0x7fa6e40045b0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 shutdown_connections 2024-08-16T19:12:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561e63207890 0x561e63209cb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561e6318c4a0 0x561e63190c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e63181be0 0x561e63194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6d37fe640 0 client.25256 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 --2- 172.21.15.62:0/3228968360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e631812c0 0x561e631946b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 >> 172.21.15.62:0/3228968360 conn(0x561e6317d850 msgr2=0x561e63197150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 shutdown_connections 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 1 -- 172.21.15.62:0/3228968360 wait complete. 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.957+0000 7fa6fcb95740 20 client.25256 trim_cache size 0 max 16384 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:12:40.960 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 10, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "3c932342": { 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 10, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 138.1, 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:12:40.961 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:40.962 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 msgr2=0x7fa6e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 0x7fa6e00a51b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009fd0 tx=0x7fa6ec004a40 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 msgr2=0x7fa6f4074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d950 tx=0x7fa6e400de10 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 shutdown_connections 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa6e00a2cf0 0x7fa6e00a51b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f41078e0 0x7fa6f4074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6f4102280 0x7fa6f4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 --2- 172.21.15.62:0/3447499208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6f41018e0 0x7fa6f4074220 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:40.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 >> 172.21.15.62:0/3447499208 conn(0x7fa6f40fd600 msgr2=0x7fa6f40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 shutdown_connections 2024-08-16T19:12:40.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:40.958+0000 7fa6fa797640 1 -- 172.21.15.62:0/3447499208 wait complete. 2024-08-16T19:12:41.650 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3c932342', quiesce took 3.2 seconds. Will release after: 14.850000000000001 2024-08-16T19:12:41.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.45 MB/sec warmup 60 sec latency 1040.550 ms 2024-08-16T19:12:42.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:41 smithi062 ceph-mon[32993]: pgmap v69: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 814 KiB/s wr, 78 op/s 2024-08-16T19:12:42.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:41 smithi155 ceph-mon[37413]: pgmap v69: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 814 KiB/s wr, 78 op/s 2024-08-16T19:12:42.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:41 smithi161 ceph-mon[37576]: pgmap v69: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 814 KiB/s wr, 78 op/s 2024-08-16T19:12:42.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.36 MB/sec warmup 61 sec latency 2040.583 ms 2024-08-16T19:12:43.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.27 MB/sec warmup 62 sec latency 3040.623 ms 2024-08-16T19:12:44.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:43 smithi062 ceph-mon[32993]: pgmap v70: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 250 KiB/s wr, 56 op/s 2024-08-16T19:12:44.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:43 smithi155 ceph-mon[37413]: pgmap v70: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 250 KiB/s wr, 56 op/s 2024-08-16T19:12:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:43 smithi161 ceph-mon[37576]: pgmap v70: 129 pgs: 129 active+clean; 46 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 250 KiB/s wr, 56 op/s 2024-08-16T19:12:44.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.19 MB/sec warmup 63 sec latency 4040.739 ms 2024-08-16T19:12:45.939 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.11 MB/sec warmup 64 sec latency 5040.864 ms 2024-08-16T19:12:46.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:45 smithi062 ceph-mon[32993]: pgmap v71: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 462 KiB/s wr, 104 op/s 2024-08-16T19:12:46.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:45 smithi155 ceph-mon[37413]: pgmap v71: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 462 KiB/s wr, 104 op/s 2024-08-16T19:12:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:45 smithi161 ceph-mon[37576]: pgmap v71: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 462 KiB/s wr, 104 op/s 2024-08-16T19:12:46.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 5.03 MB/sec warmup 65 sec latency 6040.978 ms 2024-08-16T19:12:47.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.95 MB/sec warmup 66 sec latency 7041.076 ms 2024-08-16T19:12:48.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:47 smithi062 ceph-mon[32993]: pgmap v72: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:47 smithi155 ceph-mon[37413]: pgmap v72: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:47 smithi161 ceph-mon[37576]: pgmap v72: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:48.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.88 MB/sec warmup 67 sec latency 8041.207 ms 2024-08-16T19:12:49.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.81 MB/sec warmup 68 sec latency 9041.304 ms 2024-08-16T19:12:50.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:49 smithi062 ceph-mon[32993]: pgmap v73: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:49 smithi155 ceph-mon[37413]: pgmap v73: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:50.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:49 smithi161 ceph-mon[37576]: pgmap v73: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 442 KiB/s wr, 92 op/s 2024-08-16T19:12:50.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.74 MB/sec warmup 69 sec latency 10041.348 ms 2024-08-16T19:12:51.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.67 MB/sec warmup 70 sec latency 11041.433 ms 2024-08-16T19:12:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:51 smithi062 ceph-mon[32993]: pgmap v74: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 100 op/s 2024-08-16T19:12:52.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:51 smithi155 ceph-mon[37413]: pgmap v74: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 100 op/s 2024-08-16T19:12:52.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:51 smithi161 ceph-mon[37576]: pgmap v74: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 454 KiB/s wr, 100 op/s 2024-08-16T19:12:52.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.60 MB/sec warmup 71 sec latency 12041.558 ms 2024-08-16T19:12:53.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.54 MB/sec warmup 72 sec latency 13041.664 ms 2024-08-16T19:12:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:53 smithi062 ceph-mon[32993]: pgmap v75: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-08-16T19:12:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:53 smithi155 ceph-mon[37413]: pgmap v75: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-08-16T19:12:54.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:53 smithi161 ceph-mon[37576]: pgmap v75: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 224 KiB/s wr, 56 op/s 2024-08-16T19:12:54.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.48 MB/sec warmup 73 sec latency 14041.737 ms 2024-08-16T19:12:55.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.42 MB/sec warmup 74 sec latency 15041.802 ms 2024-08-16T19:12:56.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:55 smithi062 ceph-mon[32993]: pgmap v76: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 227 KiB/s wr, 62 op/s 2024-08-16T19:12:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:55 smithi155 ceph-mon[37413]: pgmap v76: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 227 KiB/s wr, 62 op/s 2024-08-16T19:12:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:55 smithi161 ceph-mon[37576]: pgmap v76: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 227 KiB/s wr, 62 op/s 2024-08-16T19:12:56.501 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:12:56.782 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:56.940 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.36 MB/sec warmup 75 sec latency 16041.906 ms 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 -- 172.21.15.62:0/2989920461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c102bd0 msgr2=0x7fcb2c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/2989920461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c100320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009900 tx=0x7fcb1402a480 comp rx=0 tx=0).stop 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 -- 172.21.15.62:0/2989920461 shutdown_connections 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/2989920461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 0x7fcb2c110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/2989920461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c100320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/2989920461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 -- 172.21.15.62:0/2989920461 >> 172.21.15.62:0/2989920461 conn(0x7fcb2c076f40 msgr2=0x7fcb2c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 -- 172.21.15.62:0/2989920461 shutdown_connections 2024-08-16T19:12:57.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.316+0000 7fcb31e8a640 1 -- 172.21.15.62:0/2989920461 wait complete. 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 Processor -- start 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 -- start start 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 0x7fcb2c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c113f00 con 0x7fcb2c1027f0 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c113d80 con 0x7fcb2c102bd0 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb31e8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c114080 con 0x7fcb2c100940 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb2affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58120/0 (socket says 172.21.15.62:58120) 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.317+0000 7fcb2affd640 1 -- 172.21.15.62:0/4208571313 learned_addr learned my addr 172.21.15.62:0/4208571313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:57.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2b7fe640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 0x7fcb2c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2bfff640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 -- 172.21.15.62:0/4208571313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 msgr2=0x7fcb2c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 0x7fcb2c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 -- 172.21.15.62:0/4208571313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c102bd0 msgr2=0x7fcb2c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 -- 172.21.15.62:0/4208571313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c19f370 con 0x7fcb2c1027f0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb2affd640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004770 tx=0x7fcb14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1403f070 con 0x7fcb2c1027f0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c19f600 con 0x7fcb2c1027f0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14031ca0 con 0x7fcb2c1027f0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14035030 con 0x7fcb2c1027f0 2024-08-16T19:12:57.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.318+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a6f40 con 0x7fcb2c1027f0 2024-08-16T19:12:57.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.319+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c0ffde0 con 0x7fcb2c1027f0 2024-08-16T19:12:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.320+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb14042070 con 0x7fcb2c1027f0 2024-08-16T19:12:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.321+0000 7fcb28ff9640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.321+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fcb140eff20 con 0x7fcb2c1027f0 2024-08-16T19:12:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.321+0000 7fcb2b7fe640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:57.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.322+0000 7fcb2b7fe640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 0x7fcb180a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb24009680 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:57.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.325+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcb140bc160 con 0x7fcb2c1027f0 2024-08-16T19:12:57.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.536+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2c10bc40 con 0x7fcb2c1027f0 2024-08-16T19:12:57.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.537+0000 7fcb28ff9640 1 -- 172.21.15.62:0/4208571313 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11996 (secure 0 0 0) 0x7fcb140c0460 con 0x7fcb2c1027f0 2024-08-16T19:12:57.538 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:12:57.538 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-08-16T19:12:00:521513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:11:59.549161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:12:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-08-16T19:12:57.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 msgr2=0x7fcb180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:57.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 0x7fcb180a51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005fd0 tx=0x7fcb24009680 comp rx=0 tx=0).stop 2024-08-16T19:12:57.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 msgr2=0x7fcb2c19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:57.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004770 tx=0x7fcb14003c10 comp rx=0 tx=0).stop 2024-08-16T19:12:57.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 shutdown_connections 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb2c102bd0 0x7fcb2c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcb2c1027f0 0x7fcb2c19a900 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 --2- 172.21.15.62:0/4208571313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c100940 0x7fcb2c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 >> 172.21.15.62:0/4208571313 conn(0x7fcb2c076f40 msgr2=0x7fcb2c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 shutdown_connections 2024-08-16T19:12:57.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:57.539+0000 7fcb31e8a640 1 -- 172.21.15.62:0/4208571313 wait complete. 2024-08-16T19:12:57.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.30 MB/sec warmup 76 sec latency 17042.000 ms 2024-08-16T19:12:58.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:57 smithi062 ceph-mon[32993]: pgmap v77: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:12:58.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4208571313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:58.283 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 3c932342 --release --await' 2024-08-16T19:12:58.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 3c932342 --release --await 2024-08-16T19:12:58.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:57 smithi155 ceph-mon[37413]: pgmap v77: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:12:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4208571313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:58.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:57 smithi161 ceph-mon[37576]: pgmap v77: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:12:58.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4208571313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:12:58.549 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:12:58.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 57350 4.25 MB/sec warmup 77 sec latency 18042.039 ms 2024-08-16T19:12:59.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 -- 172.21.15.62:0/1732881129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201018e0 msgr2=0x7fa020101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 --2- 172.21.15.62:0/1732881129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201018e0 0x7fa020101d40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa014009930 tx=0x7fa01402a420 comp rx=0 tx=0).stop 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 -- 172.21.15.62:0/1732881129 shutdown_connections 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 --2- 172.21.15.62:0/1732881129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa020102280 0x7fa02010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 --2- 172.21.15.62:0/1732881129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201018e0 0x7fa020101d40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 --2- 172.21.15.62:0/1732881129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0201078e0 0x7fa020107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 -- 172.21.15.62:0/1732881129 >> 172.21.15.62:0/1732881129 conn(0x7fa0200fd600 msgr2=0x7fa0200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 -- 172.21.15.62:0/1732881129 shutdown_connections 2024-08-16T19:12:59.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.025+0000 7fa026234640 1 -- 172.21.15.62:0/1732881129 wait complete. 2024-08-16T19:12:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 Processor -- start 2024-08-16T19:12:59.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 -- start start 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0201018e0 0x7fa0201981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 0x7fa020198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa020111ce0 con 0x7fa0201078e0 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa020111b60 con 0x7fa0201018e0 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa026234640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa020111e60 con 0x7fa020102280 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58144/0 (socket says 172.21.15.62:58144) 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa01ffff640 1 -- 172.21.15.62:0/628197591 learned_addr learned my addr 172.21.15.62:0/628197591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.026+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0201018e0 0x7fa0201981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01effd640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 0x7fa020198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 -- 172.21.15.62:0/628197591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 msgr2=0x7fa020198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 0x7fa020198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 -- 172.21.15.62:0/628197591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0201018e0 msgr2=0x7fa0201981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0201018e0 0x7fa0201981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 -- 172.21.15.62:0/628197591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02019d190 con 0x7fa0201078e0 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01effd640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 0x7fa020198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01ffff640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa01800e970 tx=0x7fa01800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01cff9640 1 -- 172.21.15.62:0/628197591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ccc0 con 0x7fa0201078e0 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01cff9640 1 -- 172.21.15.62:0/628197591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01800ce60 con 0x7fa0201078e0 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02019d480 con 0x7fa0201078e0 2024-08-16T19:12:59.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa01cff9640 1 -- 172.21.15.62:0/628197591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018010490 con 0x7fa0201078e0 2024-08-16T19:12:59.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.027+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0201a4d60 con 0x7fa0201078e0 2024-08-16T19:12:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.029+0000 7fa01cff9640 1 -- 172.21.15.62:0/628197591 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa018005020 con 0x7fa0201078e0 2024-08-16T19:12:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 Processor -- start 2024-08-16T19:12:59.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 -- start start 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631ef124c60 con 0x5631ef1be990 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631ef0bcb90 con 0x5631ef1bf2b0 2024-08-16T19:12:59.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa028632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631ef120c20 con 0x5631ef1c9b70 2024-08-16T19:12:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01cff9640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 0x7fa0080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58158/0 (socket says 172.21.15.62:58158) 2024-08-16T19:12:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 -- 172.21.15.62:0/3345046429 learned_addr learned my addr 172.21.15.62:0/3345046429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01ffff640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 0x7fa0080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 -- 172.21.15.62:0/3345046429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 msgr2=0x5631ef1cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 -- 172.21.15.62:0/3345046429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 msgr2=0x5631ef1c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.030+0000 7fa01effd640 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa01effd640 1 -- 172.21.15.62:0/3345046429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631eee34580 con 0x5631ef1be990 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa01cff9640 1 -- 172.21.15.62:0/628197591 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa0180cfc80 con 0x7fa0201078e0 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa01effd640 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa014005940 tx=0x7fa014005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad44f52c5f676349 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa01ffff640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 0x7fa0080a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cd0f0 tx=0x7fa01809bbe0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa005ffb640 1 -- 172.21.15.62:0/3345046429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014035050 con 0x5631ef1be990 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa005ffb640 1 -- 172.21.15.62:0/3345046429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014041070 con 0x5631ef1be990 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.031+0000 7fa005ffb640 1 -- 172.21.15.62:0/3345046429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014005610 con 0x5631ef1be990 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- 172.21.15.62:0/3345046429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 msgr2=0x5631ef1bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa014005940 tx=0x7fa014005bd0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- 172.21.15.62:0/3345046429 shutdown_connections 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 --2- 172.21.15.62:0/3345046429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1be990 0x5631ef1bed70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- 172.21.15.62:0/3345046429 >> 172.21.15.62:0/3345046429 conn(0x5631ef1baf20 msgr2=0x5631ef1bb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- 172.21.15.62:0/3345046429 shutdown_connections 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- 172.21.15.62:0/3345046429 wait complete. 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 -- start start 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.032+0000 7fa028632740 1 Processor -- start 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631ef1d0b50 con 0x5631ef1c9b70 2024-08-16T19:12:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631ef0c27c0 con 0x5631ef1bf2b0 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631ef1d0cd0 con 0x5631ef1be990 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa028632740 10 client.0 _ll_register_callbacks cb 0x5631ef0cbf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:35590/0 (socket says 172.21.15.62:35590) 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01f7fe640 1 -- 172.21.15.62:0/3167698093 learned_addr learned my addr 172.21.15.62:0/3167698093 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 -- 172.21.15.62:0/3167698093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 msgr2=0x5631ef1c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 -- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 msgr2=0x5631ef1c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 -- 172.21.15.62:0/3167698093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631eeefe960 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7fa01effd640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1c7530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa014005940 tx=0x7fa014048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0cda3acf66c8ac server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.033+0000 7f9ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014035070 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa0140046c0 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa014041070 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa01405b050 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 10 client.15660 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5631ef1d0b50 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014041070 con 0x5631ef1c9b70 2024-08-16T19:12:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7fa014046a00 con 0x5631ef1c9b70 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 10 client.15660 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 20 client.15660 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef1ddd60 0x5631ef1e0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 4 client.15660 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.034+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631ef1321d0 con 0x5631ef1ddd60 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.035+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef1ddd60 0x5631ef1e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.035+0000 7f9ffeffd640 10 client.15660 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.035+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef1ddd60 0x5631ef1e0150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eebb4a14d919f06 server_cookie=3e4fc65ad590aa85 in_seq=0 out_seq=0 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3167698093 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631ef1321d0 con 0x5631ef1ddd60 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7f9ffeffd640 10 client.15660 handle_command_reply: tid=0 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 client.15660 shutdown 2024-08-16T19:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 msgr2=0x5631ef1c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1c7530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa014005940 tx=0x7fa014048040 comp rx=0 tx=0).stop 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 shutdown_connections 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef1ddd60 0x5631ef1e0150 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1c7530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 --2- 172.21.15.62:0/3167698093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 >> 172.21.15.62:0/3167698093 conn(0x5631ef1baf20 msgr2=0x5631ef1c0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7f9ffeffd640 0 client.15660 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 shutdown_connections 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 1 -- 172.21.15.62:0/3167698093 wait complete. 2024-08-16T19:12:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.036+0000 7fa028632740 20 client.15660 trim_cache size 0 max 16384 2024-08-16T19:12:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.053+0000 7fa028632740 1 Processor -- start 2024-08-16T19:12:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.053+0000 7fa028632740 1 -- start start 2024-08-16T19:12:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201acb40 con 0x5631ef1c9b70 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201ae740 con 0x5631ef1bf2b0 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa028632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201ad210 con 0x5631ef1be990 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58182/0 (socket says 172.21.15.62:58182) 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 -- 172.21.15.62:0/3579096501 learned_addr learned my addr 172.21.15.62:0/3579096501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 -- 172.21.15.62:0/3579096501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 msgr2=0x5631ef1c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 -- 172.21.15.62:0/3579096501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 msgr2=0x5631ef1c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 -- 172.21.15.62:0/3579096501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ef1b9470 con 0x5631ef1c9b70 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa01effd640 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa0140059d0 tx=0x7fa01402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1475cc0033291a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa0057fa640 1 -- 172.21.15.62:0/3579096501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014035040 con 0x5631ef1c9b70 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.054+0000 7fa0057fa640 1 -- 172.21.15.62:0/3579096501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014041070 con 0x5631ef1c9b70 2024-08-16T19:12:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa0057fa640 1 -- 172.21.15.62:0/3579096501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01403d720 con 0x5631ef1c9b70 2024-08-16T19:12:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- 172.21.15.62:0/3579096501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 msgr2=0x5631ef1e58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa0140059d0 tx=0x7fa01402a420 comp rx=0 tx=0).stop 2024-08-16T19:12:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- 172.21.15.62:0/3579096501 shutdown_connections 2024-08-16T19:12:59.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1c9b70 0x5631ef1e58e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1bf2b0 0x5631ef1c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 --2- 172.21.15.62:0/3579096501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1be990 0x5631ef1c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- 172.21.15.62:0/3579096501 >> 172.21.15.62:0/3579096501 conn(0x5631ef1baf20 msgr2=0x5631ef1c8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- 172.21.15.62:0/3579096501 shutdown_connections 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- 172.21.15.62:0/3579096501 wait complete. 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.055+0000 7fa028632740 1 -- start start 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 Processor -- start 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1be990 0x5631ef1d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201adfb0 con 0x5631ef1bf2b0 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201ad8e0 con 0x5631ef1be990 2024-08-16T19:12:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa0201aab20 con 0x5631ef1c9b70 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa028632740 10 client.0 _ll_register_callbacks cb 0x7fa014005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58186/0 (socket says 172.21.15.62:58186) 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 -- 172.21.15.62:0/3303699794 learned_addr learned my addr 172.21.15.62:0/3303699794 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01effd640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1be990 0x5631ef1d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01effd640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 -- 172.21.15.62:0/3303699794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 msgr2=0x5631ef1d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 -- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1be990 msgr2=0x5631ef1d6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1be990 0x5631ef1d6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.056+0000 7fa01f7fe640 1 -- 172.21.15.62:0/3303699794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ef1d22b0 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa01f7fe640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5631ef1e0880 tx=0x7fa010003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d78415d8e5317b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010004b20 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fa010004cc0 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa014041070 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa010013ad0 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 10 client.15666 fetch_fsmap learned FSMap version 30 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5631ef248760 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014041070 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7fa01000f040 con 0x5631ef1bf2b0 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 10 client.15666 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 10 client.15666 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 20 client.15666 populate_metadata read hostname 'smithi062' 2024-08-16T19:12:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.057+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef248fd0 0x5631ef24b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:12:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.058+0000 7fa028632740 4 client.15666 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3c932342", "release": true, "await": true}] 2024-08-16T19:12:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.058+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3c932342", "release": true, "await": true}) -- 0x5631ef12c6e0 con 0x5631ef248fd0 2024-08-16T19:12:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.058+0000 7fa01effd640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef248fd0 0x5631ef24b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:12:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.058+0000 7f9ffeffd640 10 client.15666 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.058+0000 7fa01effd640 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef248fd0 0x5631ef24b3f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9507337b436f789c server_cookie=65be32f0bd0d7ba0 in_seq=0 out_seq=0 2024-08-16T19:12:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.066+0000 7f9ffeffd640 1 -- 172.21.15.62:0/3303699794 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5631ef12c6e0 con 0x5631ef248fd0 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.066+0000 7f9ffeffd640 10 client.15666 handle_command_reply: tid=0 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.066+0000 7fa028632740 1 client.15666 shutdown 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.066+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 msgr2=0x5631ef1d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.066+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x5631ef1e0880 tx=0x7fa010003d40 comp rx=0 tx=0).stop 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 shutdown_connections 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631ef248fd0 0x5631ef24b3f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631ef1c9b70 0x5631ef1d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631ef1bf2b0 0x5631ef1d1780 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 --2- 172.21.15.62:0/3303699794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631ef1be990 0x5631ef1d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 >> 172.21.15.62:0/3303699794 conn(0x5631ef1baf20 msgr2=0x5631ef1c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7f9ffeffd640 0 client.15666 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 shutdown_connections 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 1 -- 172.21.15.62:0/3303699794 wait complete. 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa028632740 20 client.15666 trim_cache size 0 max 16384 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:12:59.069 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 12, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "3c932342": { 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 12, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 138.1, 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:12:59.070 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:12:59.071 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:12:59.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 msgr2=0x7fa0080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 0x7fa0080a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cd0f0 tx=0x7fa01809bbe0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 msgr2=0x7fa02019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa01800e970 tx=0x7fa01800ed70 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 shutdown_connections 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa0080a2d70 0x7fa0080a5230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0201078e0 0x7fa02019cab0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa020102280 0x7fa020198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 --2- 172.21.15.62:0/628197591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0201018e0 0x7fa0201981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.067+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 >> 172.21.15.62:0/628197591 conn(0x7fa0200fd600 msgr2=0x7fa0200fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.068+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 shutdown_connections 2024-08-16T19:12:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:12:59.068+0000 7fa026234640 1 -- 172.21.15.62:0/628197591 wait complete. 2024-08-16T19:12:59.924 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3c932342', total seconds elapsed: 21.5 2024-08-16T19:12:59.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 59538 4.33 MB/sec warmup 78 sec latency 18170.315 ms 2024-08-16T19:13:00.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:12:59 smithi062 ceph-mon[32993]: pgmap v78: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:13:00.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:12:59 smithi155 ceph-mon[37413]: pgmap v78: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:13:00.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:12:59 smithi161 ceph-mon[37576]: pgmap v78: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-08-16T19:13:00.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 61148 4.39 MB/sec warmup 79 sec latency 12.285 ms 2024-08-16T19:13:01.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 61861 4.39 MB/sec warmup 80 sec latency 58.304 ms 2024-08-16T19:13:02.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:01 smithi155 ceph-mon[37413]: pgmap v79: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 640 KiB/s wr, 36 op/s 2024-08-16T19:13:02.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:01 smithi062 ceph-mon[32993]: pgmap v79: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 640 KiB/s wr, 36 op/s 2024-08-16T19:13:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:01 smithi161 ceph-mon[37576]: pgmap v79: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 640 KiB/s wr, 36 op/s 2024-08-16T19:13:02.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 64285 4.51 MB/sec warmup 81 sec latency 10.506 ms 2024-08-16T19:13:03.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 66771 4.60 MB/sec warmup 82 sec latency 9.418 ms 2024-08-16T19:13:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:03 smithi062 ceph-mon[32993]: pgmap v80: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 628 KiB/s wr, 27 op/s 2024-08-16T19:13:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:03 smithi155 ceph-mon[37413]: pgmap v80: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 628 KiB/s wr, 27 op/s 2024-08-16T19:13:04.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:03 smithi161 ceph-mon[37576]: pgmap v80: 129 pgs: 129 active+clean; 54 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 628 KiB/s wr, 27 op/s 2024-08-16T19:13:04.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 68333 4.64 MB/sec warmup 83 sec latency 18.140 ms 2024-08-16T19:13:05.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 71102 4.79 MB/sec warmup 84 sec latency 8.780 ms 2024-08-16T19:13:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:05 smithi062 ceph-mon[32993]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-08-16T19:13:06.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:05 smithi155 ceph-mon[37413]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-08-16T19:13:06.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:05 smithi161 ceph-mon[37576]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-08-16T19:13:06.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 72631 4.81 MB/sec warmup 85 sec latency 13.416 ms 2024-08-16T19:13:07.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 74954 4.91 MB/sec warmup 86 sec latency 16.926 ms 2024-08-16T19:13:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:07 smithi062 ceph-mon[32993]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:08.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:07 smithi155 ceph-mon[37413]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:08.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:07 smithi161 ceph-mon[37576]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:08.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 77444 4.99 MB/sec warmup 87 sec latency 10.337 ms 2024-08-16T19:13:09.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 79020 5.03 MB/sec warmup 88 sec latency 9.170 ms 2024-08-16T19:13:10.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:09 smithi062 ceph-mon[32993]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:09 smithi155 ceph-mon[37413]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:10.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:09 smithi161 ceph-mon[37576]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-08-16T19:13:10.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 81847 5.17 MB/sec warmup 89 sec latency 9.973 ms 2024-08-16T19:13:11.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 83330 5.18 MB/sec warmup 90 sec latency 38.200 ms 2024-08-16T19:13:12.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:11 smithi161 ceph-mon[37576]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-08-16T19:13:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:11 smithi062 ceph-mon[32993]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-08-16T19:13:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:11 smithi155 ceph-mon[37413]: pgmap v84: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-08-16T19:13:12.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 85560 5.27 MB/sec warmup 91 sec latency 64.958 ms 2024-08-16T19:13:13.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 87772 5.30 MB/sec warmup 92 sec latency 11.497 ms 2024-08-16T19:13:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:13 smithi062 ceph-mon[32993]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-08-16T19:13:14.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:13 smithi155 ceph-mon[37413]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-08-16T19:13:14.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:13 smithi161 ceph-mon[37576]: pgmap v85: 129 pgs: 129 active+clean; 60 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-08-16T19:13:14.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 89447 5.37 MB/sec warmup 93 sec latency 37.239 ms 2024-08-16T19:13:15.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 92331 5.50 MB/sec warmup 94 sec latency 9.705 ms 2024-08-16T19:13:16.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:15 smithi062 ceph-mon[32993]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-08-16T19:13:16.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:15 smithi155 ceph-mon[37413]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-08-16T19:13:16.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:15 smithi161 ceph-mon[37576]: pgmap v86: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 378 op/s 2024-08-16T19:13:16.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 93775 5.51 MB/sec warmup 95 sec latency 9.441 ms 2024-08-16T19:13:17.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 96190 5.60 MB/sec warmup 96 sec latency 15.123 ms 2024-08-16T19:13:18.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:17 smithi062 ceph-mon[32993]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:18.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:17 smithi155 ceph-mon[37413]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:18.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:17 smithi161 ceph-mon[37576]: pgmap v87: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:18.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 98418 5.62 MB/sec warmup 97 sec latency 9.217 ms 2024-08-16T19:13:19.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 100084 5.68 MB/sec warmup 98 sec latency 11.227 ms 2024-08-16T19:13:20.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:19 smithi062 ceph-mon[32993]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:20.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:19 smithi155 ceph-mon[37413]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:20.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:19 smithi161 ceph-mon[37576]: pgmap v88: 129 pgs: 129 active+clean; 66 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:20.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 102988 5.80 MB/sec warmup 99 sec latency 11.250 ms 2024-08-16T19:13:21.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 104313 5.81 MB/sec warmup 100 sec latency 10.162 ms 2024-08-16T19:13:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:21 smithi062 ceph-mon[32993]: pgmap v89: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-08-16T19:13:22.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:21 smithi155 ceph-mon[37413]: pgmap v89: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-08-16T19:13:22.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:21 smithi161 ceph-mon[37576]: pgmap v89: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-08-16T19:13:22.941 INFO:tasks.workunit.client.0.smithi062.stdout: 1 106759 5.88 MB/sec warmup 101 sec latency 12.002 ms 2024-08-16T19:13:23.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 108828 5.90 MB/sec warmup 102 sec latency 13.098 ms 2024-08-16T19:13:24.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:23 smithi062 ceph-mon[32993]: pgmap v90: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:24.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:23 smithi155 ceph-mon[37413]: pgmap v90: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:24.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:23 smithi161 ceph-mon[37576]: pgmap v90: 129 pgs: 129 active+clean; 73 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 228 op/s 2024-08-16T19:13:24.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 110534 5.96 MB/sec warmup 103 sec latency 10.227 ms 2024-08-16T19:13:25.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 112980 6.02 MB/sec warmup 104 sec latency 9.504 ms 2024-08-16T19:13:26.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:25 smithi062 ceph-mon[32993]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 342 op/s 2024-08-16T19:13:26.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:25 smithi155 ceph-mon[37413]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 342 op/s 2024-08-16T19:13:26.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:25 smithi161 ceph-mon[37576]: pgmap v91: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 342 op/s 2024-08-16T19:13:26.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 114456 6.03 MB/sec warmup 105 sec latency 10.028 ms 2024-08-16T19:13:27.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 117316 6.15 MB/sec warmup 106 sec latency 9.954 ms 2024-08-16T19:13:28.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:27 smithi062 ceph-mon[32993]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:28.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:27 smithi155 ceph-mon[37413]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:27 smithi161 ceph-mon[37576]: pgmap v92: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:28.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 118897 6.14 MB/sec warmup 107 sec latency 12.026 ms 2024-08-16T19:13:29.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 121069 6.21 MB/sec warmup 108 sec latency 15.503 ms 2024-08-16T19:13:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:29 smithi062 ceph-mon[32993]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:30.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:29 smithi155 ceph-mon[37413]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:30.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:29 smithi161 ceph-mon[37576]: pgmap v93: 129 pgs: 129 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-08-16T19:13:30.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 123526 6.27 MB/sec warmup 109 sec latency 25.538 ms 2024-08-16T19:13:31.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 125066 6.28 MB/sec warmup 110 sec latency 10.900 ms 2024-08-16T19:13:32.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:31 smithi062 ceph-mon[32993]: pgmap v94: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-08-16T19:13:32.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:31 smithi155 ceph-mon[37413]: pgmap v94: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-08-16T19:13:32.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:31 smithi161 ceph-mon[37576]: pgmap v94: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 341 op/s 2024-08-16T19:13:32.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 127874 6.38 MB/sec warmup 111 sec latency 9.588 ms 2024-08-16T19:13:33.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 129250 6.38 MB/sec warmup 112 sec latency 12.396 ms 2024-08-16T19:13:34.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:33 smithi062 ceph-mon[32993]: pgmap v95: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-08-16T19:13:34.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:33 smithi155 ceph-mon[37413]: pgmap v95: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-08-16T19:13:34.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:33 smithi161 ceph-mon[37576]: pgmap v95: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 226 op/s 2024-08-16T19:13:34.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 131661 6.45 MB/sec warmup 113 sec latency 19.504 ms 2024-08-16T19:13:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:13:35.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:13:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:13:35.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:34 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:13:35.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:13:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:13:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:13:35.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:34 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:13:35.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:13:35.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:13:35.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:13:35.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:34 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:13:35.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 133726 6.45 MB/sec warmup 114 sec latency 46.468 ms 2024-08-16T19:13:36.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:35 smithi062 ceph-mon[32993]: pgmap v96: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-08-16T19:13:36.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:35 smithi155 ceph-mon[37413]: pgmap v96: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-08-16T19:13:36.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:35 smithi161 ceph-mon[37576]: pgmap v96: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 346 op/s 2024-08-16T19:13:36.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 135504 6.50 MB/sec warmup 115 sec latency 10.493 ms 2024-08-16T19:13:37.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 138208 6.57 MB/sec warmup 116 sec latency 10.608 ms 2024-08-16T19:13:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:37 smithi062 ceph-mon[32993]: pgmap v97: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:38.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:37 smithi155 ceph-mon[37413]: pgmap v97: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:38.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:37 smithi161 ceph-mon[37576]: pgmap v97: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:38.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 139620 6.57 MB/sec warmup 117 sec latency 11.920 ms 2024-08-16T19:13:39.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 142232 6.66 MB/sec warmup 118 sec latency 15.362 ms 2024-08-16T19:13:40.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:39 smithi062 ceph-mon[32993]: pgmap v98: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:40.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:39 smithi155 ceph-mon[37413]: pgmap v98: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:40.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:39 smithi161 ceph-mon[37576]: pgmap v98: 129 pgs: 129 active+clean; 92 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 232 op/s 2024-08-16T19:13:40.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 143890 6.65 MB/sec warmup 119 sec latency 31.919 ms 2024-08-16T19:13:42.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:41 smithi062 ceph-mon[32993]: pgmap v99: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 369 op/s 2024-08-16T19:13:42.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:41 smithi155 ceph-mon[37413]: pgmap v99: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 369 op/s 2024-08-16T19:13:42.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:41 smithi161 ceph-mon[37576]: pgmap v99: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 369 op/s 2024-08-16T19:13:42.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 148472 12.59 MB/sec execute 1 sec latency 14.502 ms 2024-08-16T19:13:43.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 150108 10.56 MB/sec execute 2 sec latency 10.564 ms 2024-08-16T19:13:44.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:43 smithi062 ceph-mon[32993]: pgmap v100: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-08-16T19:13:44.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:43 smithi155 ceph-mon[37413]: pgmap v100: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-08-16T19:13:44.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:43 smithi161 ceph-mon[37576]: pgmap v100: 129 pgs: 129 active+clean; 98 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-08-16T19:13:44.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 152811 12.65 MB/sec execute 3 sec latency 27.479 ms 2024-08-16T19:13:45.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 154442 10.92 MB/sec execute 4 sec latency 53.982 ms 2024-08-16T19:13:46.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:45 smithi062 ceph-mon[32993]: pgmap v101: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-08-16T19:13:46.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:45 smithi155 ceph-mon[37413]: pgmap v101: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-08-16T19:13:46.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:45 smithi161 ceph-mon[37576]: pgmap v101: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-08-16T19:13:46.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 156544 11.47 MB/sec execute 5 sec latency 18.252 ms 2024-08-16T19:13:47.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 159007 11.65 MB/sec execute 6 sec latency 9.385 ms 2024-08-16T19:13:48.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:47 smithi062 ceph-mon[32993]: pgmap v102: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:48.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:47 smithi155 ceph-mon[37413]: pgmap v102: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:48.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:47 smithi161 ceph-mon[37576]: pgmap v102: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:48.942 INFO:tasks.workunit.client.0.smithi062.stdout: 1 160583 11.04 MB/sec execute 7 sec latency 10.105 ms 2024-08-16T19:13:49.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 162651 11.13 MB/sec execute 8 sec latency 256.525 ms 2024-08-16T19:13:50.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:49 smithi062 ceph-mon[32993]: pgmap v103: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:49 smithi155 ceph-mon[37413]: pgmap v103: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:50.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:49 smithi161 ceph-mon[37576]: pgmap v103: 129 pgs: 129 active+clean; 103 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-16T19:13:50.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 164057 10.69 MB/sec execute 9 sec latency 60.642 ms 2024-08-16T19:13:51.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 166790 11.33 MB/sec execute 10 sec latency 21.256 ms 2024-08-16T19:13:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:51 smithi062 ceph-mon[32993]: pgmap v104: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-16T19:13:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:51 smithi155 ceph-mon[37413]: pgmap v104: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-16T19:13:52.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:51 smithi161 ceph-mon[37576]: pgmap v104: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 417 op/s 2024-08-16T19:13:52.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 168315 10.90 MB/sec execute 11 sec latency 9.872 ms 2024-08-16T19:13:53.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 170659 11.15 MB/sec execute 12 sec latency 17.403 ms 2024-08-16T19:13:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:53 smithi062 ceph-mon[32993]: pgmap v105: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-08-16T19:13:54.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:53 smithi155 ceph-mon[37413]: pgmap v105: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-08-16T19:13:54.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:53 smithi161 ceph-mon[37576]: pgmap v105: 129 pgs: 129 active+clean; 107 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-08-16T19:13:54.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 172856 10.91 MB/sec execute 13 sec latency 12.152 ms 2024-08-16T19:13:55.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 174374 10.93 MB/sec execute 14 sec latency 45.820 ms 2024-08-16T19:13:56.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:55 smithi062 ceph-mon[32993]: pgmap v106: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-08-16T19:13:56.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:55 smithi155 ceph-mon[37413]: pgmap v106: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-08-16T19:13:56.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:55 smithi161 ceph-mon[37576]: pgmap v106: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 418 op/s 2024-08-16T19:13:56.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 176751 11.03 MB/sec execute 15 sec latency 22.694 ms 2024-08-16T19:13:57.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 178279 10.80 MB/sec execute 16 sec latency 60.320 ms 2024-08-16T19:13:58.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:57 smithi062 ceph-mon[32993]: pgmap v107: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:13:58.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:57 smithi155 ceph-mon[37413]: pgmap v107: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:13:58.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:57 smithi161 ceph-mon[37576]: pgmap v107: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:13:58.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 181050 11.17 MB/sec execute 17 sec latency 21.320 ms 2024-08-16T19:13:59.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 182688 10.92 MB/sec execute 18 sec latency 9.695 ms 2024-08-16T19:14:00.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:13:59 smithi062 ceph-mon[32993]: pgmap v108: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:14:00.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:13:59 smithi155 ceph-mon[37413]: pgmap v108: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:14:00.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:13:59 smithi161 ceph-mon[37576]: pgmap v108: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 275 op/s 2024-08-16T19:14:00.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 184546 10.99 MB/sec execute 19 sec latency 72.144 ms 2024-08-16T19:14:01.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 185949 10.77 MB/sec execute 20 sec latency 14.789 ms 2024-08-16T19:14:02.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:01 smithi062 ceph-mon[32993]: pgmap v109: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2024-08-16T19:14:02.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:01 smithi155 ceph-mon[37413]: pgmap v109: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2024-08-16T19:14:02.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:01 smithi161 ceph-mon[37576]: pgmap v109: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2024-08-16T19:14:02.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 188313 10.91 MB/sec execute 21 sec latency 11.898 ms 2024-08-16T19:14:03.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 190353 10.74 MB/sec execute 22 sec latency 16.202 ms 2024-08-16T19:14:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:03 smithi062 ceph-mon[32993]: pgmap v110: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-08-16T19:14:04.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:03 smithi155 ceph-mon[37413]: pgmap v110: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-08-16T19:14:04.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:03 smithi161 ceph-mon[37576]: pgmap v110: 129 pgs: 129 active+clean; 117 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-08-16T19:14:04.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 192091 10.81 MB/sec execute 23 sec latency 16.431 ms 2024-08-16T19:14:05.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 194441 10.85 MB/sec execute 24 sec latency 29.477 ms 2024-08-16T19:14:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:05 smithi062 ceph-mon[32993]: pgmap v111: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-08-16T19:14:06.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:05 smithi155 ceph-mon[37413]: pgmap v111: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-08-16T19:14:06.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:05 smithi161 ceph-mon[37576]: pgmap v111: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 435 op/s 2024-08-16T19:14:06.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 195877 10.72 MB/sec execute 25 sec latency 20.912 ms 2024-08-16T19:14:06.968 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:14:06.968 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-08-16T19:14:06.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:14:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:07.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3910297049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc100b50 msgr2=0x7fc7bc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:07.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3910297049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc100b50 0x7fc7bc100f30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009900 tx=0x7fc7b802a480 comp rx=0 tx=0).stop 2024-08-16T19:14:07.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3910297049 shutdown_connections 2024-08-16T19:14:07.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3910297049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3910297049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3910297049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc100b50 0x7fc7bc100f30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3910297049 >> 172.21.15.62:0/3910297049 conn(0x7fc7bc0fc500 msgr2=0x7fc7bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3910297049 shutdown_connections 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3910297049 wait complete. 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.716+0000 7fc7c34c7640 1 Processor -- start 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 -- start start 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc100b50 0x7fc7bc197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc110cb0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc110b30 con 0x7fc7bc102c20 2024-08-16T19:14:07.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c34c7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc110e30 con 0x7fc7bc100b50 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38556/0 (socket says 172.21.15.62:38556) 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c1263640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc100b50 0x7fc7bc197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 -- 172.21.15.62:0/3313760669 learned_addr learned my addr 172.21.15.62:0/3313760669 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c1a64640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 -- 172.21.15.62:0/3313760669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc100b50 msgr2=0x7fc7bc197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc100b50 0x7fc7bc197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 -- 172.21.15.62:0/3313760669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 msgr2=0x7fc7bc19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 -- 172.21.15.62:0/3313760669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc19c0d0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c1a64640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:07.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.717+0000 7fc7c0a62640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00e9f0 tx=0x7fc7ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.718+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00cd80 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.718+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc19c3c0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.718+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ac010910 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.718+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0188f0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.718+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1a3ca0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.719+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc100f30 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.720+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc7ac010ac0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.721+0000 7fc7b27fc640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 0x7fc7a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.721+0000 7fc7c1263640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 0x7fc7a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.721+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fc7ac0cfc20 con 0x7fc7bc1022a0 2024-08-16T19:14:07.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.721+0000 7fc7c1263640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 0x7fc7a80a51b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009230 tx=0x7fc7b80058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.725+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc7ac09c4b0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 198450 10.85 MB/sec execute 26 sec latency 58.771 ms 2024-08-16T19:14:07.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.977+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7bc108dc0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.978+0000 7fc7b27fc640 1 -- 172.21.15.62:0/3313760669 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11996 (secure 0 0 0) 0x7fc7ac09bdb0 con 0x7fc7bc1022a0 2024-08-16T19:14:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:14:07.979 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"btime":"2024-08-16T19:12:00:521513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:11:59.549161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:14:07.980 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-08-16T19:14:07.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.980+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 msgr2=0x7fc7a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:07.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.980+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 0x7fc7a80a51b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009230 tx=0x7fc7b80058d0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.980+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 msgr2=0x7fc7bc197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.980+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00e9f0 tx=0x7fc7ac00eeb0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 shutdown_connections 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc7a80a2cf0 0x7fc7a80a51b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc102c20 0x7fc7bc19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7bc1022a0 0x7fc7bc197660 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 --2- 172.21.15.62:0/3313760669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7bc100b50 0x7fc7bc197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 >> 172.21.15.62:0/3313760669 conn(0x7fc7bc0fc500 msgr2=0x7fc7bc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 shutdown_connections 2024-08-16T19:14:07.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:07.981+0000 7fc7c34c7640 1 -- 172.21.15.62:0/3313760669 wait complete. 2024-08-16T19:14:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:07 smithi062 ceph-mon[32993]: pgmap v112: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:07 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3313760669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:08.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:07 smithi155 ceph-mon[37413]: pgmap v112: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:08.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:07 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3313760669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:08.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:07 smithi161 ceph-mon[37576]: pgmap v112: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:07 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3313760669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:08.840 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 139.45 --await-for 45.0' 2024-08-16T19:14:08.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 139.45 --await-for 45.0 2024-08-16T19:14:08.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 199860 10.70 MB/sec execute 27 sec latency 17.286 ms 2024-08-16T19:14:09.120 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:09.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 -- 172.21.15.62:0/2560952996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c102c20 msgr2=0x7f7f4c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 --2- 172.21.15.62:0/2560952996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c102c20 0x7f7f4c10d770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ca0 tx=0x7f7f4402a950 comp rx=0 tx=0).stop 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 -- 172.21.15.62:0/2560952996 shutdown_connections 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 --2- 172.21.15.62:0/2560952996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c102c20 0x7f7f4c10d770 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 --2- 172.21.15.62:0/2560952996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 --2- 172.21.15.62:0/2560952996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c100b50 0x7f7f4c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 -- 172.21.15.62:0/2560952996 >> 172.21.15.62:0/2560952996 conn(0x7f7f4c0fc500 msgr2=0x7f7f4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 -- 172.21.15.62:0/2560952996 shutdown_connections 2024-08-16T19:14:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.844+0000 7f7f523a6640 1 -- 172.21.15.62:0/2560952996 wait complete. 2024-08-16T19:14:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.845+0000 7f7f523a6640 1 Processor -- start 2024-08-16T19:14:09.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.845+0000 7f7f523a6640 1 -- start start 2024-08-16T19:14:09.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 0x7f7f4c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c110cb0 con 0x7f7f4c1022a0 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c110b30 con 0x7f7f4c100b50 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f523a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c110e30 con 0x7f7f4c102c20 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54862/0 (socket says 172.21.15.62:54862) 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 -- 172.21.15.62:0/290519557 learned_addr learned my addr 172.21.15.62:0/290519557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f50943640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 0x7f7f4c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 -- 172.21.15.62:0/290519557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 msgr2=0x7f7f4c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 0x7f7f4c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 -- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 msgr2=0x7f7f4c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 -- 172.21.15.62:0/290519557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19f420 con 0x7f7f4c100b50 2024-08-16T19:14:09.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f50943640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 0x7f7f4c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.846+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00e9f0 tx=0x7f7f3c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.847+0000 7f7f497fa640 1 -- 172.21.15.62:0/290519557 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00cd80 con 0x7f7f4c100b50 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.847+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19f710 con 0x7f7f4c100b50 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.847+0000 7f7f497fa640 1 -- 172.21.15.62:0/290519557 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c005500 con 0x7f7f4c100b50 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.847+0000 7f7f497fa640 1 -- 172.21.15.62:0/290519557 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c010690 con 0x7f7f4c100b50 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.847+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a6ff0 con 0x7f7f4c100b50 2024-08-16T19:14:09.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.849+0000 7f7f497fa640 1 -- 172.21.15.62:0/290519557 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f3c0046c0 con 0x7f7f4c100b50 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.849+0000 7f7f547a4740 1 Processor -- start 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.849+0000 7f7f547a4740 1 -- start start 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0fead0 0x55c94f0feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f10c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c94f05b810 con 0x55c94f0ff3f0 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c94f002c70 con 0x55c94f109cb0 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f547a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c94f0577d0 con 0x55c94f0fead0 2024-08-16T19:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38576/0 (socket says 172.21.15.62:38576) 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 -- 172.21.15.62:0/63036609 learned_addr learned my addr 172.21.15.62:0/63036609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f50943640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0fead0 0x55c94f0feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f10c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f497fa640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 0x7f7f340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f50943640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 0x7f7f340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f497fa640 1 -- 172.21.15.62:0/290519557 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f7f3c0cfc20 con 0x7f7f4c100b50 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 -- 172.21.15.62:0/63036609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0fead0 msgr2=0x55c94f0feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0fead0 0x55c94f0feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 -- 172.21.15.62:0/63036609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 msgr2=0x55c94f10c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f10c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4bfff640 1 -- 172.21.15.62:0/63036609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94edcdba0 con 0x55c94f0ff3f0 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.850+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:09.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.851+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c09ba50 tx=0x7f7f3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2cd86035b0b17f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.851+0000 7f7f327fc640 1 -- 172.21.15.62:0/63036609 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c09b890 con 0x55c94f0ff3f0 2024-08-16T19:14:09.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.851+0000 7f7f327fc640 1 -- 172.21.15.62:0/63036609 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c09c450 con 0x55c94f0ff3f0 2024-08-16T19:14:09.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.851+0000 7f7f327fc640 1 -- 172.21.15.62:0/63036609 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c09b6b0 con 0x55c94f0ff3f0 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.851+0000 7f7f50943640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 0x7f7f340a51b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f44004af0 tx=0x7f7f4402a950 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- 172.21.15.62:0/63036609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 msgr2=0x55c94f109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c09ba50 tx=0x7f7f3c004a40 comp rx=0 tx=0).stop 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- 172.21.15.62:0/63036609 shutdown_connections 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f109770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 --2- 172.21.15.62:0/63036609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0fead0 0x55c94f0feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- 172.21.15.62:0/63036609 >> 172.21.15.62:0/63036609 conn(0x55c94f0fb060 msgr2=0x55c94f0fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- 172.21.15.62:0/63036609 shutdown_connections 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- 172.21.15.62:0/63036609 wait complete. 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 -- start start 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.852+0000 7f7f547a4740 1 Processor -- start 2024-08-16T19:14:09.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0ff3f0 0x55c94f107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f1075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c94f10c880 con 0x55c94f0fead0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55c94f0169a0 con 0x55c94f109cb0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c94f10ca00 con 0x55c94f0ff3f0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f547a4740 10 client.0 _ll_register_callbacks cb 0x55c94f004c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38578/0 (socket says 172.21.15.62:38578) 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 -- 172.21.15.62:0/3943118553 learned_addr learned my addr 172.21.15.62:0/3943118553 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f50943640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f1075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0ff3f0 0x55c94f107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 -- 172.21.15.62:0/3943118553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0ff3f0 msgr2=0x55c94f107080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0ff3f0 0x55c94f107080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 -- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 msgr2=0x55c94f1075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f1075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 -- 172.21.15.62:0/3943118553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94f022220 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f50943640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f1075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0cff10 tx=0x7f7f3c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556a449411d0ecfe server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.853+0000 7f7f2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c09b6b0 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f7f3c0a4530 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c09c450 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c097ff0 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f547a4740 10 client.15684 fetch_fsmap learned FSMap version 30 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c94f1184f0 con 0x55c94f0fead0 2024-08-16T19:14:09.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.854+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c09c450 con 0x55c94f0fead0 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f7f3c0b2070 con 0x55c94f0fead0 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 10 client.15684 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 10 client.15684 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 20 client.15684 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f119840 0x55c94f11bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 4 client.15684 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c94f068f40 con 0x55c94f119840 2024-08-16T19:14:09.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f119840 0x55c94f11bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f119840 0x55c94f11bc30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4f7f3c7d1a8185 server_cookie=a3b9241f11030317 in_seq=0 out_seq=0 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.855+0000 7f7f2b7fe640 10 client.15684 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/3943118553 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c94f068f40 con 0x55c94f119840 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f2b7fe640 10 client.15684 handle_command_reply: tid=0 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 client.15684 shutdown 2024-08-16T19:14:09.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 msgr2=0x55c94f106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0cff10 tx=0x7f7f3c00c840 comp rx=0 tx=0).stop 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 shutdown_connections 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f119840 0x55c94f11bc30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f109cb0 0x55c94f1075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f2b7fe640 0 client.15684 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f0ff3f0 0x55c94f107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 --2- 172.21.15.62:0/3943118553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f106b40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 >> 172.21.15.62:0/3943118553 conn(0x55c94f0fb060 msgr2=0x55c94f0f1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.856+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 shutdown_connections 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.857+0000 7f7f547a4740 1 -- 172.21.15.62:0/3943118553 wait complete. 2024-08-16T19:14:09.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.857+0000 7f7f547a4740 20 client.15684 trim_cache size 0 max 16384 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 Processor -- start 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 -- start start 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0fead0 0x55c94f105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f1218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1acbd0 con 0x55c94f0ff3f0 2024-08-16T19:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae7d0 con 0x55c94f0fead0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f547a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad2a0 con 0x55c94f109cb0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38594/0 (socket says 172.21.15.62:38594) 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f4b7fe640 1 -- 172.21.15.62:0/1391431160 learned_addr learned my addr 172.21.15.62:0/1391431160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.876+0000 7f7f50943640 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f1218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0fead0 0x55c94f105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 -- 172.21.15.62:0/1391431160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 msgr2=0x55c94f1218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f1218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 -- 172.21.15.62:0/1391431160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0fead0 msgr2=0x55c94f105410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0fead0 0x55c94f105410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 -- 172.21.15.62:0/1391431160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94f0fbb20 con 0x55c94f0ff3f0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cfd7a67f703948 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f31ffb640 1 -- 172.21.15.62:0/1391431160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40005220 con 0x55c94f0ff3f0 2024-08-16T19:14:09.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f31ffb640 1 -- 172.21.15.62:0/1391431160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40004c80 con 0x55c94f0ff3f0 2024-08-16T19:14:09.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.877+0000 7f7f31ffb640 1 -- 172.21.15.62:0/1391431160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400108d0 con 0x55c94f0ff3f0 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 -- 172.21.15.62:0/1391431160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 msgr2=0x55c94f1213a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40005f90 comp rx=0 tx=0).stop 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 -- 172.21.15.62:0/1391431160 shutdown_connections 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f1218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0ff3f0 0x55c94f1213a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 --2- 172.21.15.62:0/1391431160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0fead0 0x55c94f105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 -- 172.21.15.62:0/1391431160 >> 172.21.15.62:0/1391431160 conn(0x55c94f0fb060 msgr2=0x55c94f0f1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 -- 172.21.15.62:0/1391431160 shutdown_connections 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.878+0000 7f7f547a4740 1 -- 172.21.15.62:0/1391431160 wait complete. 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.879+0000 7f7f547a4740 1 -- start start 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.879+0000 7f7f547a4740 1 Processor -- start 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.879+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0ff3f0 0x55c94f10e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f10e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae040 con 0x55c94f0fead0 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad970 con 0x55c94f0ff3f0 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aabd0 con 0x55c94f109cb0 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 10 client.0 _ll_register_callbacks cb 0x7f7f3c09e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38606/0 (socket says 172.21.15.62:38606) 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 -- 172.21.15.62:0/447543130 learned_addr learned my addr 172.21.15.62:0/447543130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0ff3f0 0x55c94f10e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f50943640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f10e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 -- 172.21.15.62:0/447543130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 msgr2=0x55c94f10e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f10e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 -- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0ff3f0 msgr2=0x55c94f10e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0ff3f0 0x55c94f10e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 -- 172.21.15.62:0/447543130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94f105af0 con 0x55c94f0fead0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f4bfff640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0049d0 tx=0x7f7f3c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4038fad5b90167e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c09b890 con 0x55c94f0fead0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f7f3c0a4730 con 0x55c94f0fead0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.880+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f40004c80 con 0x55c94f0fead0 2024-08-16T19:14:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c0ae050 con 0x55c94f0fead0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 10 client.15690 fetch_fsmap learned FSMap version 30 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c94f105af0 con 0x55c94f0fead0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40004c80 con 0x55c94f0fead0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 30) ==== 8386+0+0 (secure 0 0 0) 0x7f7f3c0b2040 con 0x55c94f0fead0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 10 client.15690 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 20 client.15690 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f1849c0 0x55c94f186de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 4 client.15690 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.45, "await_for": 45.0}] 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.45, "await_for": 45.0}) -- 0x55c94f063290 con 0x55c94f1849c0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.881+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f1849c0 0x55c94f186de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.882+0000 7f7f2b7fe640 10 client.15690 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:09.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.882+0000 7f7f4b7fe640 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f1849c0 0x55c94f186de0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00f433d93d499a6 server_cookie=5cf1f08b2c0850c3 in_seq=0 out_seq=0 2024-08-16T19:14:09.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 10.91 MB/sec execute 28 sec latency 66.580 ms 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.965+0000 7f7f2b7fe640 1 -- 172.21.15.62:0/447543130 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c94f063290 con 0x55c94f1849c0 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.965+0000 7f7f2b7fe640 10 client.15690 handle_command_reply: tid=0 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.965+0000 7f7f547a4740 1 client.15690 shutdown 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.965+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 msgr2=0x55c94f10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.965+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0049d0 tx=0x7f7f3c00c7d0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 shutdown_connections 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55c94f1849c0 0x55c94f186de0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c94f109cb0 0x55c94f10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c94f0ff3f0 0x55c94f10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 --2- 172.21.15.62:0/447543130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c94f0fead0 0x55c94f10dad0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 >> 172.21.15.62:0/447543130 conn(0x55c94f0fb060 msgr2=0x55c94f0f1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f2b7fe640 0 client.15690 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 shutdown_connections 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 1 -- 172.21.15.62:0/447543130 wait complete. 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.966+0000 7f7f547a4740 20 client.15690 trim_cache size 0 max 16384 2024-08-16T19:14:09.968 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 30, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 14, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "c238c40e": { 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 14, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:14:09.969 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:09.970 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 msgr2=0x7f7f340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 0x7f7f340a51b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f44004af0 tx=0x7f7f4402a950 comp rx=0 tx=0).stop 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 msgr2=0x7f7f4c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00e9f0 tx=0x7f7f3c00edf0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 shutdown_connections 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f340a2cf0 0x7f7f340a51b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f4c102c20 0x7f7f4c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f4c1022a0 0x7f7f4c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 --2- 172.21.15.62:0/290519557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c100b50 0x7f7f4c19a470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 >> 172.21.15.62:0/290519557 conn(0x7f7f4c0fc500 msgr2=0x7f7f4c069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 shutdown_connections 2024-08-16T19:14:09.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:09.967+0000 7f7f523a6640 1 -- 172.21.15.62:0/290519557 wait complete. 2024-08-16T19:14:10.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:09 smithi062 ceph-mon[32993]: pgmap v113: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:10.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:09 smithi155 ceph-mon[37413]: pgmap v113: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:09 smithi161 ceph-mon[37576]: pgmap v113: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-08-16T19:14:10.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 10.54 MB/sec execute 29 sec latency 1066.631 ms 2024-08-16T19:14:11.129 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c238c40e', quiesce took 4.2 seconds. Will release after: 15.25 2024-08-16T19:14:11.943 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 10.19 MB/sec execute 30 sec latency 2066.712 ms 2024-08-16T19:14:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:11 smithi155 ceph-mon[37413]: pgmap v114: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-16T19:14:12.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:11 smithi161 ceph-mon[37576]: pgmap v114: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-16T19:14:12.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:11 smithi062 ceph-mon[32993]: pgmap v114: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-08-16T19:14:12.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 9.86 MB/sec execute 31 sec latency 3066.827 ms 2024-08-16T19:14:13.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 9.55 MB/sec execute 32 sec latency 4066.961 ms 2024-08-16T19:14:14.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:14 smithi155 ceph-mon[37413]: pgmap v115: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-08-16T19:14:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:14 smithi161 ceph-mon[37576]: pgmap v115: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-08-16T19:14:14.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:14 smithi062 ceph-mon[32993]: pgmap v115: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-08-16T19:14:14.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 9.26 MB/sec execute 33 sec latency 5067.026 ms 2024-08-16T19:14:15.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 8.99 MB/sec execute 34 sec latency 6067.067 ms 2024-08-16T19:14:16.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:16 smithi155 ceph-mon[37413]: pgmap v116: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-08-16T19:14:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:16 smithi161 ceph-mon[37576]: pgmap v116: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-08-16T19:14:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:16 smithi062 ceph-mon[32993]: pgmap v116: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-08-16T19:14:16.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 8.73 MB/sec execute 35 sec latency 7067.178 ms 2024-08-16T19:14:17.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 8.49 MB/sec execute 36 sec latency 8067.328 ms 2024-08-16T19:14:18.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:18 smithi155 ceph-mon[37413]: pgmap v117: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:18 smithi161 ceph-mon[37576]: pgmap v117: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:18.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:18 smithi062 ceph-mon[32993]: pgmap v117: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:18.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 8.26 MB/sec execute 37 sec latency 9067.419 ms 2024-08-16T19:14:19.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 8.04 MB/sec execute 38 sec latency 10067.539 ms 2024-08-16T19:14:20.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:20 smithi062 ceph-mon[32993]: pgmap v118: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:20.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:20 smithi155 ceph-mon[37413]: pgmap v118: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:20.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:20 smithi161 ceph-mon[37576]: pgmap v118: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-08-16T19:14:20.944 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 7.84 MB/sec execute 39 sec latency 11067.597 ms 2024-08-16T19:14:21.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 7.64 MB/sec execute 40 sec latency 12067.679 ms 2024-08-16T19:14:22.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:22 smithi155 ceph-mon[37413]: pgmap v119: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-08-16T19:14:22.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:22 smithi161 ceph-mon[37576]: pgmap v119: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-08-16T19:14:22.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:22 smithi062 ceph-mon[32993]: pgmap v119: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2024-08-16T19:14:22.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 7.45 MB/sec execute 41 sec latency 13068.033 ms 2024-08-16T19:14:23.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 7.28 MB/sec execute 42 sec latency 14068.140 ms 2024-08-16T19:14:24.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:24 smithi155 ceph-mon[37413]: pgmap v120: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-08-16T19:14:24.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:24 smithi161 ceph-mon[37576]: pgmap v120: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-08-16T19:14:24.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:24 smithi062 ceph-mon[32993]: pgmap v120: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-08-16T19:14:24.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 7.11 MB/sec execute 43 sec latency 15068.256 ms 2024-08-16T19:14:25.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 6.95 MB/sec execute 44 sec latency 16068.313 ms 2024-08-16T19:14:26.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:26 smithi155 ceph-mon[37413]: pgmap v121: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:14:26.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:26 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:26.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:14:26.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:26 smithi161 ceph-mon[37576]: pgmap v121: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:14:26.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:26 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:26.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:26 smithi062 ceph-mon[32993]: pgmap v121: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:14:26.407 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:26 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:26.669 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:26.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 6.79 MB/sec execute 45 sec latency 17068.359 ms 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.136+0000 7f1bbb25b640 1 -- 172.21.15.62:0/1938040106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41018c0 msgr2=0x7f1bb4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/1938040106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41018c0 0x7f1bb4101d20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009930 tx=0x7f1ba002a420 comp rx=0 tx=0).stop 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 -- 172.21.15.62:0/1938040106 shutdown_connections 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/1938040106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 0x7f1bb410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/1938040106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41018c0 0x7f1bb4101d20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/1938040106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41078c0 0x7f1bb4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 -- 172.21.15.62:0/1938040106 >> 172.21.15.62:0/1938040106 conn(0x7f1bb40fd5e0 msgr2=0x7f1bb40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 -- 172.21.15.62:0/1938040106 shutdown_connections 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 -- 172.21.15.62:0/1938040106 wait complete. 2024-08-16T19:14:27.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 Processor -- start 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.137+0000 7f1bbb25b640 1 -- start start 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41018c0 0x7f1bb4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 0x7f1bb4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb4111d60 con 0x7f1bb41078c0 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb4111be0 con 0x7f1bb41018c0 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb4111ee0 con 0x7f1bb4102260 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb8ff7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41018c0 0x7f1bb4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57774/0 (socket says 172.21.15.62:57774) 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 -- 172.21.15.62:0/2549910091 learned_addr learned my addr 172.21.15.62:0/2549910091 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:27.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb3fff640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 0x7f1bb4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 -- 172.21.15.62:0/2549910091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 msgr2=0x7f1bb4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 0x7f1bb4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 -- 172.21.15.62:0/2549910091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41018c0 msgr2=0x7f1bb4074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41018c0 0x7f1bb4074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 -- 172.21.15.62:0/2549910091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb41a67f0 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb97f8640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b970 tx=0x7f1bac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac019070 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb41a6ae0 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.138+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac014440 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.139+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac0075d0 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.139+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb41a6ff0 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.140+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb4101da0 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.141+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1bac01c070 con 0x7f1bb41078c0 2024-08-16T19:14:27.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.141+0000 7f1bb1ffb640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 0x7f1ba40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.142+0000 7f1bb8ff7640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 0x7f1ba40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.142+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f1bac0cf090 con 0x7f1bb41078c0 2024-08-16T19:14:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.142+0000 7f1bb8ff7640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 0x7f1ba40a5230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009990 tx=0x7f1ba8009110 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.145+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1bac09b3e0 con 0x7f1bb41078c0 2024-08-16T19:14:27.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.356+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bb4075750 con 0x7f1bb41078c0 2024-08-16T19:14:27.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.356+0000 7f1bb1ffb640 1 -- 172.21.15.62:0/2549910091 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11994 (secure 0 0 0) 0x7f1bb4075750 con 0x7f1bb41078c0 2024-08-16T19:14:27.357 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:14:27.357 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"btime":"2024-08-16T19:14:25:016924+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:14:24.552282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:14:27.357 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 31 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.358+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 msgr2=0x7f1ba40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.358+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 0x7f1ba40a5230 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009990 tx=0x7f1ba8009110 comp rx=0 tx=0).stop 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.358+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 msgr2=0x7f1bb4074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.358+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b970 tx=0x7f1bac00bd70 comp rx=0 tx=0).stop 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 shutdown_connections 2024-08-16T19:14:27.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba40a2d70 0x7f1ba40a5230 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bb41078c0 0x7f1bb4074ca0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb4102260 0x7f1bb4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 --2- 172.21.15.62:0/2549910091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41018c0 0x7f1bb4074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 >> 172.21.15.62:0/2549910091 conn(0x7f1bb40fd5e0 msgr2=0x7f1bb40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 shutdown_connections 2024-08-16T19:14:27.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:27.359+0000 7f1bbb25b640 1 -- 172.21.15.62:0/2549910091 wait complete. 2024-08-16T19:14:27.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 6.64 MB/sec execute 46 sec latency 18068.395 ms 2024-08-16T19:14:28.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:28 smithi062 ceph-mon[32993]: pgmap v122: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:28.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:28 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2549910091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:28.290 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id c238c40e --release --await' 2024-08-16T19:14:28.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id c238c40e --release --await 2024-08-16T19:14:28.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:28 smithi155 ceph-mon[37413]: pgmap v122: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:28 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2549910091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:28 smithi161 ceph-mon[37576]: pgmap v122: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:28 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2549910091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:28.606 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:28.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 202395 6.50 MB/sec execute 47 sec latency 19068.501 ms 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1002904698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28102c20 msgr2=0x7f2f2810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1002904698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28102c20 0x7f2f2810d770 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f2002a950 comp rx=0 tx=0).stop 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1002904698 shutdown_connections 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1002904698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28102c20 0x7f2f2810d770 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1002904698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f281026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1002904698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28100b50 0x7f2f28100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1002904698 >> 172.21.15.62:0/1002904698 conn(0x7f2f280fc500 msgr2=0x7f2f280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1002904698 shutdown_connections 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1002904698 wait complete. 2024-08-16T19:14:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.102+0000 7f2f2d8bb640 1 Processor -- start 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 -- start start 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f28197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 0x7f2f2819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28110da0 con 0x7f2f28100b50 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28110c20 con 0x7f2f281022a0 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f2d8bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28110f20 con 0x7f2f28102c20 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57796/0 (socket says 172.21.15.62:57796) 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 -- 172.21.15.62:0/1471404010 learned_addr learned my addr 172.21.15.62:0/1471404010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:29.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f267fc640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f28197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f277fe640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 0x7f2f2819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 -- 172.21.15.62:0/1471404010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 msgr2=0x7f2f2819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 0x7f2f2819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.103+0000 7f2f26ffd640 1 -- 172.21.15.62:0/1471404010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 msgr2=0x7f2f28197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f28197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f26ffd640 1 -- 172.21.15.62:0/1471404010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2819c1d0 con 0x7f2f28100b50 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f267fc640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f28197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f277fe640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 0x7f2f2819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b4a0 tx=0x7f2f1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f2c8b9640 1 -- 172.21.15.62:0/1471404010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18014070 con 0x7f2f28100b50 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f2c8b9640 1 -- 172.21.15.62:0/1471404010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f18010af0 con 0x7f2f28100b50 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2819c4c0 con 0x7f2f28100b50 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f2c8b9640 1 -- 172.21.15.62:0/1471404010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18017d10 con 0x7f2f28100b50 2024-08-16T19:14:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.104+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f281a3da0 con 0x7f2f28100b50 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.107+0000 7f2f2c8b9640 1 -- 172.21.15.62:0/1471404010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2f18017070 con 0x7f2f28100b50 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.107+0000 7f2f2fcb9740 1 Processor -- start 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.107+0000 7f2f2fcb9740 1 -- start start 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d9f0132c60 con 0x55d9f01cc990 2024-08-16T19:14:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9f00cab90 con 0x55d9f01cd2b0 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d9f012ec20 con 0x55d9f01d7b70 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57810/0 (socket says 172.21.15.62:57810) 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 -- 172.21.15.62:0/4100075061 learned_addr learned my addr 172.21.15.62:0/4100075061 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f267fc640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2c8b9640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 0x7f2f100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 -- 172.21.15.62:0/4100075061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 msgr2=0x55d9f01d9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 -- 172.21.15.62:0/4100075061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 msgr2=0x55d9f01d7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 -- 172.21.15.62:0/4100075061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9efe42580 con 0x55d9f01cc990 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f267fc640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f2c8b9640 1 -- 172.21.15.62:0/1471404010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f2f180cebb0 con 0x7f2f28100b50 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 0x7f2f100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.108+0000 7f2f277fe640 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2f2002f5b0 tx=0x7f2f2002f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d93f71364361ec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f0dffb640 1 -- 172.21.15.62:0/4100075061 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20040040 con 0x55d9f01cc990 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f0dffb640 1 -- 172.21.15.62:0/4100075061 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f20004520 con 0x55d9f01cc990 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f0dffb640 1 -- 172.21.15.62:0/4100075061 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20034070 con 0x55d9f01cc990 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f277fe640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 0x7f2f100a51b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f2004b000 tx=0x7f2f2004c000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/4100075061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 msgr2=0x55d9f01ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2f2002f5b0 tx=0x7f2f2002f5e0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/4100075061 shutdown_connections 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/4100075061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01ccd70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/4100075061 >> 172.21.15.62:0/4100075061 conn(0x55d9f01c8f20 msgr2=0x55d9f01c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/4100075061 shutdown_connections 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.109+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/4100075061 wait complete. 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 -- start start 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 Processor -- start 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d9f01dc950 con 0x55d9f01cc990 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55d9f00d07c0 con 0x55d9f01cd2b0 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d9f01dcad0 con 0x55d9f01d7b70 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f2fcb9740 10 client.0 _ll_register_callbacks cb 0x55d9f00d9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.110+0000 7f2f26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57826/0 (socket says 172.21.15.62:57826) 2024-08-16T19:14:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 -- 172.21.15.62:0/909376963 learned_addr learned my addr 172.21.15.62:0/909376963 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f277fe640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f267fc640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 -- 172.21.15.62:0/909376963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 msgr2=0x55d9f01d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 -- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 msgr2=0x55d9f01d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 -- 172.21.15.62:0/909376963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eff0c960 con 0x55d9f01cc990 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800be60 tx=0x7f2f1809a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888976ab4f782606 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801e070 con 0x55d9f01cc990 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f2f18097460 con 0x55d9f01cc990 2024-08-16T19:14:29.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f20004520 con 0x55d9f01cc990 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f1809f1a0 con 0x55d9f01cc990 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f2fcb9740 10 client.15708 fetch_fsmap learned FSMap version 31 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9f01e89a0 con 0x55d9f01cc990 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.111+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f20004520 con 0x55d9f01cc990 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f2f1809a240 con 0x55d9f01cc990 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 10 client.15708 fetch_fsmap finished waiting for FSMap version 31 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 10 client.15708 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 20 client.15708 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f01e9f00 0x55d9f01ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 4 client.15708 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9f01401d0 con 0x55d9f01e9f00 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f267fc640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f01e9f00 0x55d9f01ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f267fc640 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f01e9f00 0x55d9f01ec2f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90eb7f97665fc87d server_cookie=22154dc3c0f42b45 in_seq=0 out_seq=0 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.112+0000 7f2f06ffd640 10 client.15708 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f06ffd640 1 -- 172.21.15.62:0/909376963 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9f01401d0 con 0x55d9f01e9f00 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f06ffd640 10 client.15708 handle_command_reply: tid=0 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 client.15708 shutdown 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 msgr2=0x55d9f01d4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800be60 tx=0x7f2f1809a810 comp rx=0 tx=0).stop 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 shutdown_connections 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f01e9f00 0x55d9f01ec2f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01d7b70 0x55d9f01d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/909376963 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cc990 0x55d9f01d4a10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f06ffd640 0 client.15708 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 >> 172.21.15.62:0/909376963 conn(0x55d9f01c8f20 msgr2=0x55d9f01cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 shutdown_connections 2024-08-16T19:14:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.113+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/909376963 wait complete. 2024-08-16T19:14:29.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.114+0000 7f2f2fcb9740 20 client.15708 trim_cache size 0 max 16384 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 Processor -- start 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 -- start start 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01d7b70 0x55d9f01f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281accc0 con 0x55d9f01cd2b0 2024-08-16T19:14:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ae890 con 0x55d9f01d7b70 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ad390 con 0x55d9f01cc990 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57834/0 (socket says 172.21.15.62:57834) 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 -- 172.21.15.62:0/597524334 learned_addr learned my addr 172.21.15.62:0/597524334 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 -- 172.21.15.62:0/597524334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 msgr2=0x55d9f01cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 -- 172.21.15.62:0/597524334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01d7b70 msgr2=0x55d9f01f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01d7b70 0x55d9f01f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 -- 172.21.15.62:0/597524334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f01ec970 con 0x55d9f01cd2b0 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.131+0000 7f2f267fc640 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55d9f01d42a0 tx=0x7f2f1c016430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2d9926508e6d2a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f0d7fa640 1 -- 172.21.15.62:0/597524334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c016b90 con 0x55d9f01cd2b0 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f0d7fa640 1 -- 172.21.15.62:0/597524334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c01f030 con 0x55d9f01cd2b0 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f0d7fa640 1 -- 172.21.15.62:0/597524334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c014610 con 0x55d9f01cd2b0 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/597524334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 msgr2=0x55d9f01f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55d9f01d42a0 tx=0x7f2f1c016430 comp rx=0 tx=0).stop 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/597524334 shutdown_connections 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01d7b70 0x55d9f01f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01cd2b0 0x55d9f01f1c00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/597524334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/597524334 >> 172.21.15.62:0/597524334 conn(0x55d9f01c8f20 msgr2=0x55d9f01d6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/597524334 shutdown_connections 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.132+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/597524334 wait complete. 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 -- start start 2024-08-16T19:14:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 Processor -- start 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ae130 con 0x55d9f01d7b70 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ada60 con 0x55d9f01cd2b0 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281aaca0 con 0x55d9f01cc990 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f2fcb9740 10 client.0 _ll_register_callbacks cb 0x7f2f1809a930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f26ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f277fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57840/0 (socket says 172.21.15.62:57840) 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.133+0000 7f2f277fe640 1 -- 172.21.15.62:0/2531795303 learned_addr learned my addr 172.21.15.62:0/2531795303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f267fc640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 -- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 msgr2=0x55d9f01e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 -- 172.21.15.62:0/2531795303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 msgr2=0x55d9f01decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 -- 172.21.15.62:0/2531795303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f01c9bb0 con 0x55d9f01cc990 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f277fe640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f267fc640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f26ffd640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01e1d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b470 tx=0x7f2f18000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3590ff26aea78c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1801e070 con 0x55d9f01cc990 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f2f18010ca0 con 0x55d9f01cc990 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.134+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1c01f030 con 0x55d9f01cc990 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2f18096b40 con 0x55d9f01cc990 2024-08-16T19:14:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 10 client.25342 fetch_fsmap learned FSMap version 31 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d9f01c8d50 con 0x55d9f01cc990 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c01f030 con 0x55d9f01cc990 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f2f18061020 con 0x55d9f01cc990 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 10 client.25342 fetch_fsmap finished waiting for FSMap version 31 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 10 client.25342 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 20 client.25342 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f0255840 0x55d9f0257c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 4 client.25342 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c238c40e", "release": true, "await": true}] 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c238c40e", "release": true, "await": true}) -- 0x55d9f013a6e0 con 0x55d9f0255840 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.135+0000 7f2f267fc640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f0255840 0x55d9f0257c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.136+0000 7f2f267fc640 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f0255840 0x55d9f0257c60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797d881bdaf170f5 server_cookie=f02e0396f9271726 in_seq=0 out_seq=0 2024-08-16T19:14:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.136+0000 7f2f06ffd640 10 client.25342 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f06ffd640 1 -- 172.21.15.62:0/2531795303 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d9f013a6e0 con 0x55d9f0255840 2024-08-16T19:14:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f06ffd640 10 client.25342 handle_command_reply: tid=0 2024-08-16T19:14:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 client.25342 shutdown 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 msgr2=0x55d9f01e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01e1d20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b470 tx=0x7f2f18000bc0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 shutdown_connections 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55d9f0255840 0x55d9f0257c60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d9f01d7b70 0x55d9f01decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9f01cd2b0 0x55d9f01e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f06ffd640 0 client.25342 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 --2- 172.21.15.62:0/2531795303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d9f01cc990 0x55d9f01e1d20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 >> 172.21.15.62:0/2531795303 conn(0x55d9f01c8f20 msgr2=0x55d9f01c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.145+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 shutdown_connections 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2fcb9740 1 -- 172.21.15.62:0/2531795303 wait complete. 2024-08-16T19:14:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2fcb9740 20 client.25342 trim_cache size 0 max 16384 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 31, 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 16, 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "c238c40e": { 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 16, 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:14:29.148 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:29.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:29.150 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:29.150 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:14:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 msgr2=0x7f2f100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 0x7f2f100a51b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f2004b000 tx=0x7f2f2004c000 comp rx=0 tx=0).stop 2024-08-16T19:14:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 msgr2=0x7f2f28197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b4a0 tx=0x7f2f1800b960 comp rx=0 tx=0).stop 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 shutdown_connections 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2f100a2cf0 0x7f2f100a51b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f28102c20 0x7f2f2819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f281022a0 0x7f2f28197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 --2- 172.21.15.62:0/1471404010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f28100b50 0x7f2f28197220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 >> 172.21.15.62:0/1471404010 conn(0x7f2f280fc500 msgr2=0x7f2f280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 shutdown_connections 2024-08-16T19:14:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:29.146+0000 7f2f2d8bb640 1 -- 172.21.15.62:0/1471404010 wait complete. 2024-08-16T19:14:29.893 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c238c40e', total seconds elapsed: 22.9 2024-08-16T19:14:29.893 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-08-16T19:14:29.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:14:29.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 203321 6.42 MB/sec execute 48 sec latency 19272.587 ms 2024-08-16T19:14:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:30 smithi062 ceph-mon[32993]: pgmap v123: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:30.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:30 smithi155 ceph-mon[37413]: pgmap v123: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:30.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:30 smithi161 ceph-mon[37576]: pgmap v123: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-08-16T19:14:30.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.674+0000 7fc0c19eb640 1 -- 172.21.15.62:0/2090879779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc100b50 msgr2=0x7fc0bc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.674+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/2090879779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc100b50 0x7fc0bc100f30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009900 tx=0x7fc0ac02a480 comp rx=0 tx=0).stop 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 -- 172.21.15.62:0/2090879779 shutdown_connections 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/2090879779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc102c20 0x7fc0bc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/2090879779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/2090879779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc100b50 0x7fc0bc100f30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 -- 172.21.15.62:0/2090879779 >> 172.21.15.62:0/2090879779 conn(0x7fc0bc0fc500 msgr2=0x7fc0bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 -- 172.21.15.62:0/2090879779 shutdown_connections 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 -- 172.21.15.62:0/2090879779 wait complete. 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 Processor -- start 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.675+0000 7fc0c19eb640 1 -- start start 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:30.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc102c20 0x7fc0bc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc110cb0 con 0x7fc0bc1022a0 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc110b30 con 0x7fc0bc102c20 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0c19eb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc0bc110e30 con 0x7fc0bc100b50 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:57848/0 (socket says 172.21.15.62:57848) 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:45796/0 (socket says 172.21.15.62:45796) 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0ba7fc640 1 -- 172.21.15.62:0/625185194 learned_addr learned my addr 172.21.15.62:0/625185194 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0bb7fe640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc102c20 0x7fc0bc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 -- 172.21.15.62:0/625185194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc102c20 msgr2=0x7fc0bc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc102c20 0x7fc0bc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 -- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 msgr2=0x7fc0bc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:30.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0baffd640 1 -- 172.21.15.62:0/625185194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0bc19d1e0 con 0x7fc0bc100b50 2024-08-16T19:14:30.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.676+0000 7fc0ba7fc640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0baffd640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac005940 tx=0x7fc0ac003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac03f070 con 0x7fc0bc100b50 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0bc19d470 con 0x7fc0bc100b50 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac031c30 con 0x7fc0bc100b50 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac035030 con 0x7fc0bc100b50 2024-08-16T19:14:30.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.677+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0bc1a4db0 con 0x7fc0bc100b50 2024-08-16T19:14:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.678+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0bc101ee0 con 0x7fc0bc100b50 2024-08-16T19:14:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.679+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc0ac042070 con 0x7fc0bc100b50 2024-08-16T19:14:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.680+0000 7fc0c09e9640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 0x7fc0a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.680+0000 7fc0ba7fc640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 0x7fc0a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:30.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.680+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fc0ac0f0770 con 0x7fc0bc100b50 2024-08-16T19:14:30.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.681+0000 7fc0ba7fc640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 0x7fc0a40a51b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0ffc70 tx=0x7fc0b0005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:30.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.683+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc0ac031de0 con 0x7fc0bc100b50 2024-08-16T19:14:30.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.916+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0bc100f30 con 0x7fc0bc100b50 2024-08-16T19:14:30.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.917+0000 7fc0c09e9640 1 -- 172.21.15.62:0/625185194 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11989 (secure 0 0 0) 0x7fc0ac0bc950 con 0x7fc0bc100b50 2024-08-16T19:14:30.918 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:14:30.918 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"btime":"2024-08-16T19:14:30:028029+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:14:29.551449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:14:30.919 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 32 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 msgr2=0x7fc0a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 0x7fc0a40a51b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0ffc70 tx=0x7fc0b0005f90 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 msgr2=0x7fc0bc198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac005940 tx=0x7fc0ac003c10 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 shutdown_connections 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc0a40a2cf0 0x7fc0a40a51b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0bc102c20 0x7fc0bc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0bc1022a0 0x7fc0bc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 --2- 172.21.15.62:0/625185194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0bc100b50 0x7fc0bc198230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 >> 172.21.15.62:0/625185194 conn(0x7fc0bc0fc500 msgr2=0x7fc0bc069f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 shutdown_connections 2024-08-16T19:14:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:30.920+0000 7fc0c19eb640 1 -- 172.21.15.62:0/625185194 wait complete. 2024-08-16T19:14:30.945 INFO:tasks.workunit.client.0.smithi062.stdout: 1 205970 6.63 MB/sec execute 49 sec latency 16.095 ms 2024-08-16T19:14:31.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:31 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:31.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:31 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/625185194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:31.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:31 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:31.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:31 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/625185194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:31.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:31 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:31.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:31 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/625185194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:31.638 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 139.45 --await-for 45.0' 2024-08-16T19:14:31.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 139.45 --await-for 45.0 2024-08-16T19:14:31.934 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:31.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 207493 6.61 MB/sec execute 50 sec latency 44.982 ms 2024-08-16T19:14:32.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:32 smithi155 ceph-mon[37413]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-16T19:14:32.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:32 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:32.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:32 smithi062 ceph-mon[32993]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-16T19:14:32.354 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:32 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:32.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:32 smithi161 ceph-mon[37576]: pgmap v124: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-16T19:14:32.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:32 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 -- 172.21.15.62:0/1182315474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc102c20 msgr2=0x7f55dc10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 --2- 172.21.15.62:0/1182315474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc102c20 0x7f55dc10d770 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009ca0 tx=0x7f55d402a950 comp rx=0 tx=0).stop 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 -- 172.21.15.62:0/1182315474 shutdown_connections 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 --2- 172.21.15.62:0/1182315474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc102c20 0x7f55dc10d770 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 --2- 172.21.15.62:0/1182315474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc1022a0 0x7f55dc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 --2- 172.21.15.62:0/1182315474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc100b50 0x7f55dc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 -- 172.21.15.62:0/1182315474 >> 172.21.15.62:0/1182315474 conn(0x7f55dc0fc500 msgr2=0x7f55dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 -- 172.21.15.62:0/1182315474 shutdown_connections 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.451+0000 7f55e11bd640 1 -- 172.21.15.62:0/1182315474 wait complete. 2024-08-16T19:14:32.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 Processor -- start 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 -- start start 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 0x7f55dc0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc110cb0 con 0x7f55dc1022a0 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc110b30 con 0x7f55dc100b50 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.452+0000 7f55e11bd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc110e30 con 0x7f55dc102c20 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55db59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 0x7f55dc0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50844/0 (socket says 172.21.15.62:50844) 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 -- 172.21.15.62:0/765115973 learned_addr learned my addr 172.21.15.62:0/765115973 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47556/0 (socket says 172.21.15.62:47556) 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 -- 172.21.15.62:0/765115973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 msgr2=0x7f55dc0fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 0x7f55dc0fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 -- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 msgr2=0x7f55dc0fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 -- 172.21.15.62:0/765115973 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1aabe0 con 0x7f55dc100b50 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55db59e640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 0x7f55dc0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55da59c640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55dad9d640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9f0 tx=0x7f55cc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.453+0000 7f55c3fff640 1 -- 172.21.15.62:0/765115973 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00cd80 con 0x7f55dc100b50 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.454+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1aae10 con 0x7f55dc100b50 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.454+0000 7f55c3fff640 1 -- 172.21.15.62:0/765115973 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55cc005500 con 0x7f55dc100b50 2024-08-16T19:14:32.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.454+0000 7f55c3fff640 1 -- 172.21.15.62:0/765115973 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc010690 con 0x7f55dc100b50 2024-08-16T19:14:32.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.454+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1ab270 con 0x7f55dc100b50 2024-08-16T19:14:32.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.456+0000 7f55c3fff640 1 -- 172.21.15.62:0/765115973 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55cc0046c0 con 0x7f55dc100b50 2024-08-16T19:14:32.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.456+0000 7f55e35bb740 1 Processor -- start 2024-08-16T19:14:32.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 -- start start 2024-08-16T19:14:32.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b614a0 0x565029b63890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565029abb810 con 0x565029b614a0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565029a62c70 con 0x565029b56be0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55e35bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565029ab77d0 con 0x565029b562c0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55c3fff640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 0x7f55c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55db59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55db59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47358/0 (socket says 172.21.15.62:47358) 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50852/0 (socket says 172.21.15.62:50852) 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55db59e640 1 -- 172.21.15.62:0/1346351751 learned_addr learned my addr 172.21.15.62:0/1346351751 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55db59e640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 0x7f55c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55c3fff640 1 -- 172.21.15.62:0/765115973 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55cc0cfbf0 con 0x7f55dc100b50 2024-08-16T19:14:32.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.457+0000 7f55da59c640 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b614a0 0x565029b63890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 -- 172.21.15.62:0/1346351751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 msgr2=0x565029b60f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 -- 172.21.15.62:0/1346351751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b614a0 msgr2=0x565029b63890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b614a0 0x565029b63890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 -- 172.21.15.62:0/1346351751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502982dba0 con 0x565029b562c0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55dad9d640 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009430 tx=0x7f55d402f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d63fbd0a8c31d37 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55db59e640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 0x7f55c40a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009ca0 tx=0x7f55d4043000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55c17fa640 1 -- 172.21.15.62:0/1346351751 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4034040 con 0x565029b562c0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55c17fa640 1 -- 172.21.15.62:0/1346351751 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d404e070 con 0x565029b562c0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.458+0000 7f55c17fa640 1 -- 172.21.15.62:0/1346351751 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4005050 con 0x565029b562c0 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- 172.21.15.62:0/1346351751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 msgr2=0x565029b566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009430 tx=0x7f55d402f830 comp rx=0 tx=0).stop 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- 172.21.15.62:0/1346351751 shutdown_connections 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b614a0 0x565029b63890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b60f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 --2- 172.21.15.62:0/1346351751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b566a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- 172.21.15.62:0/1346351751 >> 172.21.15.62:0/1346351751 conn(0x565029b52850 msgr2=0x565029b52c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- 172.21.15.62:0/1346351751 shutdown_connections 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- 172.21.15.62:0/1346351751 wait complete. 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 -- start start 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.459+0000 7f55e35bb740 1 Processor -- start 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b5ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b614a0 0x565029b5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565029b63fb0 con 0x565029b562c0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x565029a769a0 con 0x565029b56be0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x565029b64130 con 0x565029b614a0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55e35bb740 10 client.0 _ll_register_callbacks cb 0x565029a64c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b5ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47576/0 (socket says 172.21.15.62:47576) 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55dad9d640 1 -- 172.21.15.62:0/2294737067 learned_addr learned my addr 172.21.15.62:0/2294737067 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55db59e640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b614a0 0x565029b5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 -- 172.21.15.62:0/2294737067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b614a0 msgr2=0x565029b5ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b614a0 0x565029b5ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 -- 172.21.15.62:0/2294737067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 msgr2=0x565029b5e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55da59c640 1 -- 172.21.15.62:0/2294737067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565029a82220 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.460+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55da59c640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b5ea20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0fe170 tx=0x7f55d000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2cd8aa4dd9108 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0012110 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55d0004050 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d404e070 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55d00122b0 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55e35bb740 10 client.25337 fetch_fsmap learned FSMap version 33 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565029b6fdc0 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d404e070 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f55d0010430 con 0x565029b56be0 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55e35bb740 10 client.25337 fetch_fsmap finished waiting for FSMap version 33 2024-08-16T19:14:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.461+0000 7f55e35bb740 10 client.25337 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55e35bb740 20 client.25337 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029b71280 0x565029b73670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55e35bb740 4 client.25337 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:14:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565029ac8f40 con 0x565029b71280 2024-08-16T19:14:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029b71280 0x565029b73670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55beff5640 10 client.25337 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.462+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029b71280 0x565029b73670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6395b663e59d51a7 server_cookie=1edcf0d77b1c461a in_seq=0 out_seq=0 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55beff5640 1 -- 172.21.15.62:0/2294737067 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565029ac8f40 con 0x565029b71280 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55beff5640 10 client.25337 handle_command_reply: tid=0 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 client.25337 shutdown 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 msgr2=0x565029b5ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b5ea20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0fe170 tx=0x7f55d000f790 comp rx=0 tx=0).stop 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 shutdown_connections 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029b71280 0x565029b73670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b614a0 0x565029b5ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b56be0 0x565029b5ea20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2294737067 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b562c0 0x565029b5e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55beff5640 0 client.25337 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 >> 172.21.15.62:0/2294737067 conn(0x565029b52850 msgr2=0x565029b51b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 shutdown_connections 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 1 -- 172.21.15.62:0/2294737067 wait complete. 2024-08-16T19:14:32.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.463+0000 7f55e35bb740 20 client.25337 trim_cache size 0 max 16384 2024-08-16T19:14:32.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 Processor -- start 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 -- start start 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b5b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b5410 con 0x565029b56be0 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b6f80 con 0x565029b614a0 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55e35bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b5ae0 con 0x565029b562c0 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47580/0 (socket says 172.21.15.62:47580) 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 -- 172.21.15.62:0/3428682235 learned_addr learned my addr 172.21.15.62:0/3428682235 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:32.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55db59e640 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b5b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 -- 172.21.15.62:0/3428682235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 msgr2=0x565029b78f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b78f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 -- 172.21.15.62:0/3428682235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 msgr2=0x565029b5b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b5b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.483+0000 7f55da59c640 1 -- 172.21.15.62:0/3428682235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565029b6d210 con 0x565029b56be0 2024-08-16T19:14:32.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.484+0000 7f55da59c640 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f55d0000c00 tx=0x7f55d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d72758b553eeac server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.484+0000 7f55c0ff9640 1 -- 172.21.15.62:0/3428682235 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0010430 con 0x565029b56be0 2024-08-16T19:14:32.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.484+0000 7f55c0ff9640 1 -- 172.21.15.62:0/3428682235 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d00105d0 con 0x565029b56be0 2024-08-16T19:14:32.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.484+0000 7f55c0ff9640 1 -- 172.21.15.62:0/3428682235 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0003920 con 0x565029b56be0 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- 172.21.15.62:0/3428682235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 msgr2=0x565029b79480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f55d0000c00 tx=0x7f55d0009f90 comp rx=0 tx=0).stop 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- 172.21.15.62:0/3428682235 shutdown_connections 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b5b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b79480 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 --2- 172.21.15.62:0/3428682235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b78f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- 172.21.15.62:0/3428682235 >> 172.21.15.62:0/3428682235 conn(0x565029b52850 msgr2=0x565029b52c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- 172.21.15.62:0/3428682235 shutdown_connections 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- 172.21.15.62:0/3428682235 wait complete. 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 -- start start 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.485+0000 7f55e35bb740 1 Processor -- start 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b68b0 con 0x565029b56be0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b61b0 con 0x565029b614a0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1b33f0 con 0x565029b562c0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55e35bb740 10 client.0 _ll_register_callbacks cb 0x7f55d0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55da59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55db59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55db59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b65cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50882/0 (socket says 172.21.15.62:50882) 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55db59e640 1 -- 172.21.15.62:0/2942806350 learned_addr learned my addr 172.21.15.62:0/2942806350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 -- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 msgr2=0x565029b65cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b65cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 -- 172.21.15.62:0/2942806350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 msgr2=0x565029b657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 -- 172.21.15.62:0/2942806350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565029bdb0b0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55da59c640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55dad9d640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b65270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9c0 tx=0x7f55cc0cfe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3723403afa0f7de9 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.486+0000 7f55beff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0ca040 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f55cc0cd1d0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d00105d0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55cc0cd3d0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 10 client.25366 fetch_fsmap learned FSMap version 33 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565029bdc3d0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d00105d0 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f55cc09b170 con 0x565029b562c0 2024-08-16T19:14:32.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 10 client.25366 fetch_fsmap finished waiting for FSMap version 33 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 10 client.25366 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 20 client.25366 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029bdcb10 0x565029bdef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 4 client.25366 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.45, "await_for": 45.0}] 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.487+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.45, "await_for": 45.0}) -- 0x565029ac3290 con 0x565029bdcb10 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.488+0000 7f55da59c640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029bdcb10 0x565029bdef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.488+0000 7f55da59c640 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029bdcb10 0x565029bdef00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e085ccc2de787029 server_cookie=e626126f0273df4c in_seq=0 out_seq=0 2024-08-16T19:14:32.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.488+0000 7f55beff5640 10 client.25366 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55beff5640 1 -- 172.21.15.62:0/2942806350 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565029ac3290 con 0x565029bdcb10 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55beff5640 10 client.25366 handle_command_reply: tid=0 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 client.25366 shutdown 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 msgr2=0x565029b65270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b65270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9c0 tx=0x7f55cc0cfe70 comp rx=0 tx=0).stop 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 shutdown_connections 2024-08-16T19:14:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565029bdcb10 0x565029bdef00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565029b614a0 0x565029b65cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565029b56be0 0x565029b657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 --2- 172.21.15.62:0/2942806350 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x565029b562c0 0x565029b65270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55beff5640 0 client.25366 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 >> 172.21.15.62:0/2942806350 conn(0x565029b52850 msgr2=0x565029b51310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 shutdown_connections 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 1 -- 172.21.15.62:0/2942806350 wait complete. 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.558+0000 7f55e35bb740 20 client.25366 trim_cache size 0 max 16384 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:14:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 33, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 18, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "d522897b": { 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 18, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:14:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:32.562 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 msgr2=0x7f55c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 0x7f55c40a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009ca0 tx=0x7f55d4043000 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 msgr2=0x7f55dc0fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9f0 tx=0x7f55cc00edf0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 shutdown_connections 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f55c40a2cf0 0x7f55c40a51b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55dc102c20 0x7f55dc0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55dc1022a0 0x7f55dc0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 --2- 172.21.15.62:0/765115973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55dc100b50 0x7f55dc0fc9b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 >> 172.21.15.62:0/765115973 conn(0x7f55dc0fc500 msgr2=0x7f55dc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:32.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 shutdown_connections 2024-08-16T19:14:32.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:32.559+0000 7f55e11bd640 1 -- 172.21.15.62:0/765115973 wait complete. 2024-08-16T19:14:32.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.63 MB/sec execute 51 sec latency 463.225 ms 2024-08-16T19:14:33.392 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd522897b', quiesce took 3.5 seconds. Will release after: 15.95 2024-08-16T19:14:33.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.51 MB/sec execute 52 sec latency 1463.338 ms 2024-08-16T19:14:34.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:34 smithi155 ceph-mon[37413]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-08-16T19:14:34.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:34 smithi161 ceph-mon[37576]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-08-16T19:14:34.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:34 smithi062 ceph-mon[32993]: pgmap v125: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-08-16T19:14:34.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.38 MB/sec execute 53 sec latency 2463.446 ms 2024-08-16T19:14:35.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:35 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:14:35.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:35 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:14:35.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:35 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:14:35.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.27 MB/sec execute 54 sec latency 3463.561 ms 2024-08-16T19:14:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:36 smithi155 ceph-mon[37413]: pgmap v126: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:14:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:14:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:14:36.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:36 smithi161 ceph-mon[37576]: pgmap v126: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:36.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:14:36.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:14:36.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:14:36.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:36 smithi062 ceph-mon[32993]: pgmap v126: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:36.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:14:36.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:14:36.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:14:36.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.15 MB/sec execute 55 sec latency 4463.639 ms 2024-08-16T19:14:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:37 smithi062 ceph-mon[32993]: pgmap v127: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:37.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:37 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:37.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:37 smithi155 ceph-mon[37413]: pgmap v127: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:37 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:37 smithi161 ceph-mon[37576]: pgmap v127: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:37 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:14:37.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 6.04 MB/sec execute 56 sec latency 5463.689 ms 2024-08-16T19:14:38.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.94 MB/sec execute 57 sec latency 6463.809 ms 2024-08-16T19:14:39.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:39 smithi062 ceph-mon[32993]: pgmap v128: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:39.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:39 smithi155 ceph-mon[37413]: pgmap v128: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:39 smithi161 ceph-mon[37576]: pgmap v128: 129 pgs: 129 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 150 op/s 2024-08-16T19:14:39.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.83 MB/sec execute 58 sec latency 7463.900 ms 2024-08-16T19:14:40.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.73 MB/sec execute 59 sec latency 8464.013 ms 2024-08-16T19:14:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:41 smithi062 ceph-mon[32993]: pgmap v129: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:41.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:41 smithi155 ceph-mon[37413]: pgmap v129: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:41 smithi161 ceph-mon[37576]: pgmap v129: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 178 op/s 2024-08-16T19:14:41.946 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.64 MB/sec execute 60 sec latency 9464.128 ms 2024-08-16T19:14:42.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.55 MB/sec execute 61 sec latency 10464.231 ms 2024-08-16T19:14:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:43 smithi155 ceph-mon[37413]: pgmap v130: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-08-16T19:14:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:43 smithi161 ceph-mon[37576]: pgmap v130: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-08-16T19:14:43.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.46 MB/sec execute 62 sec latency 11464.366 ms 2024-08-16T19:14:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:43 smithi062 ceph-mon[32993]: pgmap v130: 129 pgs: 129 active+clean; 137 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-08-16T19:14:44.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.37 MB/sec execute 63 sec latency 12464.474 ms 2024-08-16T19:14:45.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:45 smithi155 ceph-mon[37413]: pgmap v131: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-08-16T19:14:45.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:45 smithi161 ceph-mon[37576]: pgmap v131: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-08-16T19:14:45.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.29 MB/sec execute 64 sec latency 13464.600 ms 2024-08-16T19:14:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:45 smithi062 ceph-mon[32993]: pgmap v131: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-08-16T19:14:46.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.21 MB/sec execute 65 sec latency 14464.724 ms 2024-08-16T19:14:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:47 smithi155 ceph-mon[37413]: pgmap v132: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:47 smithi161 ceph-mon[37576]: pgmap v132: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:47.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.13 MB/sec execute 66 sec latency 15464.833 ms 2024-08-16T19:14:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:47 smithi062 ceph-mon[32993]: pgmap v132: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:48.947 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 5.05 MB/sec execute 67 sec latency 16464.932 ms 2024-08-16T19:14:49.343 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:14:49.620 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:49.742 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:49 smithi062 ceph-mon[32993]: pgmap v133: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:49 smithi155 ceph-mon[37413]: pgmap v133: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:49 smithi161 ceph-mon[37576]: pgmap v133: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-08-16T19:14:49.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 4.98 MB/sec execute 68 sec latency 17473.610 ms 2024-08-16T19:14:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- 172.21.15.62:0/4014208352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 msgr2=0x7f99341078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:50.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4014208352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f99341078c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9924009930 tx=0x7f992402a420 comp rx=0 tx=0).stop 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- 172.21.15.62:0/4014208352 shutdown_connections 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4014208352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934107fd0 0x7f993410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4014208352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f99341078c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4014208352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934102a60 0x7f9934102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- 172.21.15.62:0/4014208352 >> 172.21.15.62:0/4014208352 conn(0x7f99340fc500 msgr2=0x7f99340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- 172.21.15.62:0/4014208352 shutdown_connections 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- 172.21.15.62:0/4014208352 wait complete. 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 Processor -- start 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.102+0000 7f993c5e9640 1 -- start start 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934102a60 0x7f993419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934107fd0 0x7f993419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9934111dd0 con 0x7f9934103410 2024-08-16T19:14:50.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9934111c50 con 0x7f9934102a60 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993c5e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9934111f50 con 0x7f9934107fd0 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49080/0 (socket says 172.21.15.62:49080) 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 -- 172.21.15.62:0/4040954561 learned_addr learned my addr 172.21.15.62:0/4040954561 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993a385640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934102a60 0x7f993419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f993ab86640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934107fd0 0x7f993419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 -- 172.21.15.62:0/4040954561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934107fd0 msgr2=0x7f993419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934107fd0 0x7f993419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 -- 172.21.15.62:0/4040954561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934102a60 msgr2=0x7f993419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934102a60 0x7f993419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 -- 172.21.15.62:0/4040954561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993419f480 con 0x7f9934103410 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f9939b84640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9924005940 tx=0x7f99240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:50.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.103+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992403f030 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.104+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993419f710 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.104+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9924003ce0 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.104+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a7050 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.104+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924049060 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.105+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9934102e40 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.106+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9924042030 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.107+0000 7f992b7fe640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 0x7f99200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.107+0000 7f993a385640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 0x7f99200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.107+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f99240f0190 con 0x7f9934103410 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.107+0000 7f993a385640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 0x7f99200a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f993000bbe0 tx=0x7f993000b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:50.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.114+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f99240bc3d0 con 0x7f9934103410 2024-08-16T19:14:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.327+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f993419b830 con 0x7f9934103410 2024-08-16T19:14:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.328+0000 7f992b7fe640 1 -- 172.21.15.62:0/4040954561 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11996 (secure 0 0 0) 0x7f993419b830 con 0x7f9934103410 2024-08-16T19:14:50.330 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:14:50.330 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"btime":"2024-08-16T19:14:36:483831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:14:35.575734+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:14:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 34 2024-08-16T19:14:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 msgr2=0x7f99200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 0x7f99200a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f993000bbe0 tx=0x7f993000b390 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 msgr2=0x7f993419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9924005940 tx=0x7f99240042f0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 shutdown_connections 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f99200a2cf0 0x7f99200a51b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9934107fd0 0x7f993419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9934103410 0x7f993419aa10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 --2- 172.21.15.62:0/4040954561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9934102a60 0x7f993419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 >> 172.21.15.62:0/4040954561 conn(0x7f99340fc500 msgr2=0x7f993410cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 shutdown_connections 2024-08-16T19:14:50.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:50.331+0000 7f993c5e9640 1 -- 172.21.15.62:0/4040954561 wait complete. 2024-08-16T19:14:50.662 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:50 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4040954561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:50.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:50 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4040954561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:50 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4040954561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:14:50.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 208934 4.90 MB/sec execute 69 sec latency 18473.644 ms 2024-08-16T19:14:51.109 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d522897b --release --await' 2024-08-16T19:14:51.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d522897b --release --await 2024-08-16T19:14:51.378 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:14:51.825 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:51 smithi062 ceph-mon[32993]: pgmap v134: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 -- 172.21.15.62:0/3462104034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381018b0 msgr2=0x7f6238101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 --2- 172.21.15.62:0/3462104034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381018b0 0x7f6238101d10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f622c009930 tx=0x7f622c02a420 comp rx=0 tx=0).stop 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 -- 172.21.15.62:0/3462104034 shutdown_connections 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 --2- 172.21.15.62:0/3462104034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6238102250 0x7f623810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 --2- 172.21.15.62:0/3462104034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62381018b0 0x7f6238101d10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 --2- 172.21.15.62:0/3462104034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 0x7f6238107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 -- 172.21.15.62:0/3462104034 >> 172.21.15.62:0/3462104034 conn(0x7f62380fd5f0 msgr2=0x7f62380ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 -- 172.21.15.62:0/3462104034 shutdown_connections 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.823+0000 7f623e701640 1 -- 172.21.15.62:0/3462104034 wait complete. 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 Processor -- start 2024-08-16T19:14:51.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 -- start start 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62381018b0 0x7f6238074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 0x7f6238074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6238111cb0 con 0x7f6238102250 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f6238111b30 con 0x7f62381018b0 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623e701640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6238111e30 con 0x7f62381078b0 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39114/0 (socket says 172.21.15.62:39114) 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 -- 172.21.15.62:0/1214549923 learned_addr learned my addr 172.21.15.62:0/1214549923 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f623cc9e640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 0x7f6238074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f6237fff640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62381018b0 0x7f6238074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:51 smithi155 ceph-mon[37413]: pgmap v134: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 -- 172.21.15.62:0/1214549923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 msgr2=0x7f6238074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 0x7f6238074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 -- 172.21.15.62:0/1214549923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62381018b0 msgr2=0x7f6238074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62381018b0 0x7f6238074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.825+0000 7f62377fe640 1 -- 172.21.15.62:0/1214549923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62381a6620 con 0x7f6238102250 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f62377fe640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f622c009a50 tx=0x7f622c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f62357fa640 1 -- 172.21.15.62:0/1214549923 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c03f030 con 0x7f6238102250 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62381a68b0 con 0x7f6238102250 2024-08-16T19:14:51.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f62357fa640 1 -- 172.21.15.62:0/1214549923 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c003ce0 con 0x7f6238102250 2024-08-16T19:14:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f62357fa640 1 -- 172.21.15.62:0/1214549923 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c035070 con 0x7f6238102250 2024-08-16T19:14:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.826+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62381a6d90 con 0x7f6238102250 2024-08-16T19:14:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f62357fa640 1 -- 172.21.15.62:0/1214549923 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f622c042030 con 0x7f6238102250 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 Processor -- start 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 -- start start 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cea8347c60 con 0x55cea83e1990 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cea82dfb90 con 0x55cea83ecb70 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f6240aff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cea8343c20 con 0x55cea83e22b0 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f62357fa640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 0x7f62200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f623cc9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.829+0000 7f623cc9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39124/0 (socket says 172.21.15.62:39124) 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 -- 172.21.15.62:0/3827758954 learned_addr learned my addr 172.21.15.62:0/3827758954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 0x7f62200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f6237fff640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f62357fa640 1 -- 172.21.15.62:0/1214549923 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f622c0efa30 con 0x7f6238102250 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f62377fe640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 -- 172.21.15.62:0/3827758954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 msgr2=0x55cea83ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 -- 172.21.15.62:0/3827758954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 msgr2=0x55cea83eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 -- 172.21.15.62:0/3827758954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea8057580 con 0x55cea83e1990 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f6237fff640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6230014f10 tx=0x7f62300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daae7e348123b69 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f623cc9e640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 0x7f62200a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6230005c40 tx=0x7f6230016000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f621e7fc640 1 -- 172.21.15.62:0/3827758954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230024070 con 0x55cea83e1990 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f621e7fc640 1 -- 172.21.15.62:0/3827758954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6230007ce0 con 0x55cea83e1990 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.830+0000 7f621e7fc640 1 -- 172.21.15.62:0/3827758954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230005760 con 0x55cea83e1990 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 -- 172.21.15.62:0/3827758954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 msgr2=0x55cea83e1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6230014f10 tx=0x7f62300117b0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 -- 172.21.15.62:0/3827758954 shutdown_connections 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 --2- 172.21.15.62:0/3827758954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e1990 0x55cea83e1d70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 -- 172.21.15.62:0/3827758954 >> 172.21.15.62:0/3827758954 conn(0x55cea83ddf20 msgr2=0x55cea83de300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 -- 172.21.15.62:0/3827758954 shutdown_connections 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.831+0000 7f6240aff740 1 -- 172.21.15.62:0/3827758954 wait complete. 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 -- start start 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 Processor -- start 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83ea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cea83ef740 con 0x55cea83e22b0 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cea82e57c0 con 0x55cea83ecb70 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cea83ef8c0 con 0x55cea83e1990 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6240aff740 10 client.0 _ll_register_callbacks cb 0x55cea82eef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39132/0 (socket says 172.21.15.62:39132) 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f62377fe640 1 -- 172.21.15.62:0/881402089 learned_addr learned my addr 172.21.15.62:0/881402089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.832+0000 7f6237fff640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f623cc9e640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 -- 172.21.15.62:0/881402089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 msgr2=0x55cea83e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 -- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 msgr2=0x55cea83ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 -- 172.21.15.62:0/881402089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea8121960 con 0x55cea83e22b0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62377fe640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c0bb7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269a02bef7ed331b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62177fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c049040 con 0x55cea83e22b0 2024-08-16T19:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f622c0ed070 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cea83551d0 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f622c0bff70 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f6240aff740 10 client.15756 fetch_fsmap learned FSMap version 34 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cea83fb7b0 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.833+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cea83551d0 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f622c03b460 con 0x55cea83e22b0 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 10 client.15756 fetch_fsmap finished waiting for FSMap version 34 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 10 client.15756 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 20 client.15756 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea83fcb50 0x55cea83fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 4 client.15756 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cea834f6e0 con 0x55cea83fcb50 2024-08-16T19:14:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6237fff640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea83fcb50 0x55cea83fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f6237fff640 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea83fcb50 0x55cea83fefa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4683feae6d7c38df server_cookie=b9da39cb64aaa327 in_seq=0 out_seq=0 2024-08-16T19:14:51.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.834+0000 7f62177fe640 10 client.15756 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.835+0000 7f62177fe640 1 -- 172.21.15.62:0/881402089 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cea834f6e0 con 0x55cea83fcb50 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.835+0000 7f62177fe640 10 client.15756 handle_command_reply: tid=0 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.835+0000 7f6240aff740 1 client.15756 shutdown 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.835+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 msgr2=0x55cea83e9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.835+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c0bb7c0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 shutdown_connections 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea83fcb50 0x55cea83fefa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea83ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e9fd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 --2- 172.21.15.62:0/881402089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f62177fe640 0 client.15756 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 >> 172.21.15.62:0/881402089 conn(0x55cea83ddf20 msgr2=0x55cea83dd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 shutdown_connections 2024-08-16T19:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 1 -- 172.21.15.62:0/881402089 wait complete. 2024-08-16T19:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.836+0000 7f6240aff740 20 client.15756 trim_cache size 0 max 16384 2024-08-16T19:14:51.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 Processor -- start 2024-08-16T19:14:51.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 -- start start 2024-08-16T19:14:51.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea8404860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62381b0ee0 con 0x55cea83e22b0 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62381b2ae0 con 0x55cea83ecb70 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6240aff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62381b15b0 con 0x55cea83e1990 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f62377fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39148/0 (socket says 172.21.15.62:39148) 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f62377fe640 1 -- 172.21.15.62:0/1330046505 learned_addr learned my addr 172.21.15.62:0/1330046505 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f623cc9e640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea8404860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.853+0000 7f6237fff640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 -- 172.21.15.62:0/1330046505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 msgr2=0x55cea83de300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83de300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 -- 172.21.15.62:0/1330046505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 msgr2=0x55cea8404860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea8404860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 -- 172.21.15.62:0/1330046505 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea83eb9d0 con 0x55cea83e22b0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f623cc9e640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea8404860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f62377fe640 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f622c0b8020 tx=0x7f622c0bb830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679d67c97f9dcf46 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f621dffb640 1 -- 172.21.15.62:0/1330046505 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c0ba360 con 0x55cea83e22b0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f621dffb640 1 -- 172.21.15.62:0/1330046505 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c0d2020 con 0x55cea83e22b0 2024-08-16T19:14:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.854+0000 7f621dffb640 1 -- 172.21.15.62:0/1330046505 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c03b5c0 con 0x55cea83e22b0 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- 172.21.15.62:0/1330046505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 msgr2=0x55cea83e3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f622c0b8020 tx=0x7f622c0bb830 comp rx=0 tx=0).stop 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- 172.21.15.62:0/1330046505 shutdown_connections 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83ecb70 0x55cea8404860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83e22b0 0x55cea83e3af0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 --2- 172.21.15.62:0/1330046505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e1990 0x55cea83de300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- 172.21.15.62:0/1330046505 >> 172.21.15.62:0/1330046505 conn(0x55cea83ddf20 msgr2=0x55cea83e33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- 172.21.15.62:0/1330046505 shutdown_connections 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- 172.21.15.62:0/1330046505 wait complete. 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 -- start start 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.855+0000 7f6240aff740 1 Processor -- start 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83e1990 0x55cea83f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62381b2410 con 0x55cea83ecb70 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f62381b1c80 con 0x55cea83e1990 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62381aeec0 con 0x55cea83e22b0 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6240aff740 10 client.0 _ll_register_callbacks cb 0x7f622c0bb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39150/0 (socket says 172.21.15.62:39150) 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 -- 172.21.15.62:0/835048276 learned_addr learned my addr 172.21.15.62:0/835048276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f62377fe640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f6237fff640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83e1990 0x55cea83f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 -- 172.21.15.62:0/835048276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 msgr2=0x55cea83f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 -- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83e1990 msgr2=0x55cea83f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83e1990 0x55cea83f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 -- 172.21.15.62:0/835048276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea83f21f0 con 0x55cea83ecb70 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f623cc9e640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6230014640 tx=0x7f6230014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc6dd940adf8965 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.856+0000 7f62177fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230036060 con 0x55cea83ecb70 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f6230003970 con 0x55cea83ecb70 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622c0d2020 con 0x55cea83ecb70 2024-08-16T19:14:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f623003d050 con 0x55cea83ecb70 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 10 client.15762 fetch_fsmap learned FSMap version 34 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cea8468280 con 0x55cea83ecb70 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c0d2020 con 0x55cea83ecb70 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f6230003b90 con 0x55cea83ecb70 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 10 client.15762 fetch_fsmap finished waiting for FSMap version 34 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 10 client.15762 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 20 client.15762 populate_metadata read hostname 'smithi062' 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea84686c0 0x55cea846aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 4 client.15762 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d522897b", "release": true, "await": true}] 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.857+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d522897b", "release": true, "await": true}) -- 0x55cea82cc5a0 con 0x55cea84686c0 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.858+0000 7f6237fff640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea84686c0 0x55cea846aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.858+0000 7f6237fff640 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea84686c0 0x55cea846aab0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d394ed4ade7d6233 server_cookie=189547367e97d41f in_seq=0 out_seq=0 2024-08-16T19:14:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.858+0000 7f62177fe640 10 client.15762 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:51.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f62177fe640 1 -- 172.21.15.62:0/835048276 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cea82cc5a0 con 0x55cea84686c0 2024-08-16T19:14:51.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f62177fe640 10 client.15762 handle_command_reply: tid=0 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 client.15762 shutdown 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 msgr2=0x55cea83f1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6230014640 tx=0x7f6230014e30 comp rx=0 tx=0).stop 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 shutdown_connections 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cea84686c0 0x55cea846aab0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cea83ecb70 0x55cea83f1ae0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cea83e22b0 0x55cea83f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 --2- 172.21.15.62:0/835048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cea83e1990 0x55cea83f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f62177fe640 0 client.15762 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 >> 172.21.15.62:0/835048276 conn(0x55cea83ddf20 msgr2=0x55cea83e3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 shutdown_connections 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 1 -- 172.21.15.62:0/835048276 wait complete. 2024-08-16T19:14:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.867+0000 7f6240aff740 20 client.15762 trim_cache size 0 max 16384 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 34, 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 20, 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:14:51.870 INFO:teuthology.orchestra.run.smithi062.stdout: "d522897b": { 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 20, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 139.4, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:14:51.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:14:51.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 msgr2=0x7f62200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 0x7f62200a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6230005c40 tx=0x7f6230016000 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 msgr2=0x7f6238074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f622c009a50 tx=0x7f622c004270 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 shutdown_connections 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f62200a2cf0 0x7f62200a51b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62381078b0 0x7f6238074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6238102250 0x7f6238074760 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 --2- 172.21.15.62:0/1214549923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62381018b0 0x7f6238074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 >> 172.21.15.62:0/1214549923 conn(0x7f62380fd5f0 msgr2=0x7f62380fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 shutdown_connections 2024-08-16T19:14:51.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:14:51.868+0000 7f623e701640 1 -- 172.21.15.62:0/1214549923 wait complete. 2024-08-16T19:14:51.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:51 smithi161 ceph-mon[37576]: pgmap v134: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.2 MiB/s wr, 68 op/s 2024-08-16T19:14:51.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209117 4.85 MB/sec execute 70 sec latency 19387.820 ms 2024-08-16T19:14:52.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd522897b', total seconds elapsed: 22.8 2024-08-16T19:14:52.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209256 4.82 MB/sec execute 71 sec latency 981.581 ms 2024-08-16T19:14:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:53 smithi155 ceph-mon[37413]: pgmap v135: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 517 KiB/s wr, 40 op/s 2024-08-16T19:14:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:53 smithi161 ceph-mon[37576]: pgmap v135: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 517 KiB/s wr, 40 op/s 2024-08-16T19:14:53.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209256 4.76 MB/sec execute 72 sec latency 1981.691 ms 2024-08-16T19:14:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:53 smithi062 ceph-mon[32993]: pgmap v135: 129 pgs: 129 active+clean; 127 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 517 KiB/s wr, 40 op/s 2024-08-16T19:14:54.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209256 4.69 MB/sec execute 73 sec latency 2981.739 ms 2024-08-16T19:14:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:55 smithi155 ceph-mon[37413]: pgmap v136: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 538 KiB/s wr, 48 op/s 2024-08-16T19:14:55.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:55 smithi161 ceph-mon[37576]: pgmap v136: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 538 KiB/s wr, 48 op/s 2024-08-16T19:14:55.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209347 4.63 MB/sec execute 74 sec latency 3338.721 ms 2024-08-16T19:14:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:55 smithi062 ceph-mon[32993]: pgmap v136: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 538 KiB/s wr, 48 op/s 2024-08-16T19:14:56.956 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209347 4.57 MB/sec execute 75 sec latency 1605.072 ms 2024-08-16T19:14:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:57 smithi155 ceph-mon[37413]: pgmap v137: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:14:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:57 smithi161 ceph-mon[37576]: pgmap v137: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:14:57.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209347 4.51 MB/sec execute 76 sec latency 2605.173 ms 2024-08-16T19:14:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:57 smithi062 ceph-mon[32993]: pgmap v137: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:14:58.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209347 4.45 MB/sec execute 77 sec latency 3605.260 ms 2024-08-16T19:14:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:14:59 smithi155 ceph-mon[37413]: pgmap v138: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:14:59.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:14:59 smithi161 ceph-mon[37576]: pgmap v138: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:14:59.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209347 4.39 MB/sec execute 78 sec latency 4605.360 ms 2024-08-16T19:14:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:14:59 smithi062 ceph-mon[32993]: pgmap v138: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 24 op/s 2024-08-16T19:15:00.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 209472 4.35 MB/sec execute 79 sec latency 4763.398 ms 2024-08-16T19:15:01.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:01 smithi155 ceph-mon[37413]: pgmap v139: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 31 op/s 2024-08-16T19:15:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:01 smithi161 ceph-mon[37576]: pgmap v139: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 31 op/s 2024-08-16T19:15:01.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 210946 4.37 MB/sec execute 80 sec latency 44.085 ms 2024-08-16T19:15:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:01 smithi062 ceph-mon[32993]: pgmap v139: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 70 KiB/s wr, 31 op/s 2024-08-16T19:15:02.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 213195 4.48 MB/sec execute 81 sec latency 37.170 ms 2024-08-16T19:15:03.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:03 smithi155 ceph-mon[37413]: pgmap v140: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-08-16T19:15:03.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:03 smithi161 ceph-mon[37576]: pgmap v140: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-08-16T19:15:03.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 215330 4.52 MB/sec execute 82 sec latency 24.001 ms 2024-08-16T19:15:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:03 smithi062 ceph-mon[32993]: pgmap v140: 129 pgs: 129 active+clean; 128 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-08-16T19:15:04.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 217089 4.61 MB/sec execute 83 sec latency 10.003 ms 2024-08-16T19:15:05.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:05 smithi155 ceph-mon[37413]: pgmap v141: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-08-16T19:15:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:05 smithi161 ceph-mon[37576]: pgmap v141: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-08-16T19:15:05.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 219444 4.70 MB/sec execute 84 sec latency 19.302 ms 2024-08-16T19:15:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:05 smithi062 ceph-mon[32993]: pgmap v141: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-08-16T19:15:06.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 220944 4.73 MB/sec execute 85 sec latency 16.942 ms 2024-08-16T19:15:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:07 smithi155 ceph-mon[37413]: pgmap v142: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:07 smithi161 ceph-mon[37576]: pgmap v142: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:07.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.73 MB/sec execute 86 sec latency 798.641 ms 2024-08-16T19:15:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:07 smithi062 ceph-mon[32993]: pgmap v142: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:08.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.67 MB/sec execute 87 sec latency 1798.719 ms 2024-08-16T19:15:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:09 smithi155 ceph-mon[37413]: pgmap v143: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:09 smithi161 ceph-mon[37576]: pgmap v143: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:09.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.62 MB/sec execute 88 sec latency 2798.817 ms 2024-08-16T19:15:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:09 smithi062 ceph-mon[32993]: pgmap v143: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-08-16T19:15:10.957 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.57 MB/sec execute 89 sec latency 3798.879 ms 2024-08-16T19:15:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:11 smithi155 ceph-mon[37413]: pgmap v144: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-08-16T19:15:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:11 smithi161 ceph-mon[37576]: pgmap v144: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-08-16T19:15:11.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.52 MB/sec execute 90 sec latency 4798.979 ms 2024-08-16T19:15:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:11 smithi062 ceph-mon[32993]: pgmap v144: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 206 op/s 2024-08-16T19:15:12.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.47 MB/sec execute 91 sec latency 5799.082 ms 2024-08-16T19:15:13.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:13 smithi155 ceph-mon[37413]: pgmap v145: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-16T19:15:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:13 smithi161 ceph-mon[37576]: pgmap v145: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-16T19:15:13.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.42 MB/sec execute 92 sec latency 6799.197 ms 2024-08-16T19:15:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:13 smithi062 ceph-mon[32993]: pgmap v145: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-08-16T19:15:14.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 221350 4.37 MB/sec execute 93 sec latency 7799.312 ms 2024-08-16T19:15:15.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:15 smithi155 ceph-mon[37413]: pgmap v146: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-08-16T19:15:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:15 smithi161 ceph-mon[37576]: pgmap v146: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-08-16T19:15:15.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 222738 4.39 MB/sec execute 94 sec latency 8179.562 ms 2024-08-16T19:15:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:15 smithi062 ceph-mon[32993]: pgmap v146: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-08-16T19:15:16.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 224304 4.43 MB/sec execute 95 sec latency 10.321 ms 2024-08-16T19:15:17.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:17 smithi155 ceph-mon[37413]: pgmap v147: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:17.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:17 smithi161 ceph-mon[37576]: pgmap v147: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:17.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 226535 4.51 MB/sec execute 96 sec latency 15.785 ms 2024-08-16T19:15:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:17 smithi062 ceph-mon[32993]: pgmap v147: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:18.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 227999 4.54 MB/sec execute 97 sec latency 15.425 ms 2024-08-16T19:15:19.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:19 smithi155 ceph-mon[37413]: pgmap v148: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:19 smithi161 ceph-mon[37576]: pgmap v148: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:19.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 230328 4.63 MB/sec execute 98 sec latency 16.148 ms 2024-08-16T19:15:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:19 smithi062 ceph-mon[32993]: pgmap v148: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 649 KiB/s wr, 79 op/s 2024-08-16T19:15:20.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 231733 4.66 MB/sec execute 99 sec latency 13.054 ms 2024-08-16T19:15:21.812 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:21 smithi062 ceph-mon[32993]: pgmap v149: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-08-16T19:15:21.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:21 smithi155 ceph-mon[37413]: pgmap v149: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-08-16T19:15:21.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:21 smithi161 ceph-mon[37576]: pgmap v149: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-08-16T19:15:21.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 234146 4.76 MB/sec execute 100 sec latency 15.885 ms 2024-08-16T19:15:22.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 235532 4.78 MB/sec execute 101 sec latency 16.635 ms 2024-08-16T19:15:23.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:23 smithi155 ceph-mon[37413]: pgmap v150: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 596 KiB/s wr, 92 op/s 2024-08-16T19:15:23.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:23 smithi161 ceph-mon[37576]: pgmap v150: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 596 KiB/s wr, 92 op/s 2024-08-16T19:15:23.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 237843 4.86 MB/sec execute 102 sec latency 17.052 ms 2024-08-16T19:15:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:23 smithi062 ceph-mon[32993]: pgmap v150: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 596 KiB/s wr, 92 op/s 2024-08-16T19:15:24.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 239168 4.88 MB/sec execute 103 sec latency 13.381 ms 2024-08-16T19:15:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:25 smithi155 ceph-mon[37413]: pgmap v151: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-08-16T19:15:25.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:25 smithi161 ceph-mon[37576]: pgmap v151: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-08-16T19:15:25.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 241533 4.96 MB/sec execute 104 sec latency 20.042 ms 2024-08-16T19:15:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:25 smithi062 ceph-mon[32993]: pgmap v151: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-08-16T19:15:26.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 242861 4.97 MB/sec execute 105 sec latency 13.280 ms 2024-08-16T19:15:27.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:27 smithi155 ceph-mon[37413]: pgmap v152: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:27.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:27 smithi161 ceph-mon[37576]: pgmap v152: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:27.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 245019 5.05 MB/sec execute 106 sec latency 11.684 ms 2024-08-16T19:15:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:27 smithi062 ceph-mon[32993]: pgmap v152: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:28.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 246305 5.05 MB/sec execute 107 sec latency 17.194 ms 2024-08-16T19:15:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:29 smithi155 ceph-mon[37413]: pgmap v153: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:29 smithi161 ceph-mon[37576]: pgmap v153: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:29.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 248617 5.13 MB/sec execute 108 sec latency 13.169 ms 2024-08-16T19:15:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:29 smithi062 ceph-mon[32993]: pgmap v153: 129 pgs: 129 active+clean; 152 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-08-16T19:15:30.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:30 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:30.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:30 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:30.958 INFO:tasks.workunit.client.0.smithi062.stdout: 1 250105 5.14 MB/sec execute 109 sec latency 22.743 ms 2024-08-16T19:15:30.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:30 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:31 smithi155 ceph-mon[37413]: pgmap v154: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-08-16T19:15:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:31 smithi161 ceph-mon[37576]: pgmap v154: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-08-16T19:15:31.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 252285 5.21 MB/sec execute 110 sec latency 15.654 ms 2024-08-16T19:15:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:31 smithi062 ceph-mon[32993]: pgmap v154: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 283 op/s 2024-08-16T19:15:32.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 253847 5.22 MB/sec execute 111 sec latency 32.715 ms 2024-08-16T19:15:33.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:33 smithi155 ceph-mon[37413]: pgmap v155: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-08-16T19:15:33.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:33 smithi161 ceph-mon[37576]: pgmap v155: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-08-16T19:15:33.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 255847 5.29 MB/sec execute 112 sec latency 15.260 ms 2024-08-16T19:15:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:33 smithi062 ceph-mon[32993]: pgmap v155: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 199 op/s 2024-08-16T19:15:34.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 257621 5.30 MB/sec execute 113 sec latency 13.137 ms 2024-08-16T19:15:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:35 smithi161 ceph-mon[37576]: pgmap v156: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-08-16T19:15:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:35 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:35.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 259392 5.37 MB/sec execute 114 sec latency 10.758 ms 2024-08-16T19:15:35.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:35 smithi062 ceph-mon[32993]: pgmap v156: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-08-16T19:15:35.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:35 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:36.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:35 smithi155 ceph-mon[37413]: pgmap v156: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.4 MiB/s wr, 322 op/s 2024-08-16T19:15:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:35 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:36.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:15:36.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:15:36.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:15:36.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:15:36.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 261410 5.38 MB/sec execute 115 sec latency 19.532 ms 2024-08-16T19:15:36.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:15:36.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:15:36.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:15:36.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:15:37.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:15:37.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:15:37.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:15:37.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:15:37.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 263115 5.44 MB/sec execute 116 sec latency 11.920 ms 2024-08-16T19:15:38.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:38 smithi161 ceph-mon[37576]: pgmap v157: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:38.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:38 smithi062 ceph-mon[32993]: pgmap v157: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:38.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:38 smithi155 ceph-mon[37413]: pgmap v157: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:38.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 265084 5.46 MB/sec execute 117 sec latency 30.169 ms 2024-08-16T19:15:39.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 266726 5.51 MB/sec execute 118 sec latency 18.921 ms 2024-08-16T19:15:40.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:40 smithi161 ceph-mon[37576]: pgmap v158: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:40.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:40 smithi062 ceph-mon[32993]: pgmap v158: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:40.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:40 smithi155 ceph-mon[37413]: pgmap v158: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 223 op/s 2024-08-16T19:15:40.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 268667 5.53 MB/sec execute 119 sec latency 13.771 ms 2024-08-16T19:15:41.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 270284 5.58 MB/sec execute 120 sec latency 13.475 ms 2024-08-16T19:15:42.381 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:42 smithi155 ceph-mon[37413]: pgmap v159: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-08-16T19:15:42.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:42 smithi161 ceph-mon[37576]: pgmap v159: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-08-16T19:15:42.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:42 smithi062 ceph-mon[32993]: pgmap v159: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-08-16T19:15:42.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 272502 5.63 MB/sec execute 121 sec latency 14.786 ms 2024-08-16T19:15:43.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 273942 5.65 MB/sec execute 122 sec latency 15.143 ms 2024-08-16T19:15:44.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:44 smithi161 ceph-mon[37576]: pgmap v160: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-08-16T19:15:44.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:44 smithi062 ceph-mon[32993]: pgmap v160: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-08-16T19:15:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:44 smithi155 ceph-mon[37413]: pgmap v160: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-08-16T19:15:44.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 276028 5.69 MB/sec execute 123 sec latency 15.641 ms 2024-08-16T19:15:45.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 277506 5.71 MB/sec execute 124 sec latency 12.122 ms 2024-08-16T19:15:46.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:46 smithi161 ceph-mon[37576]: pgmap v161: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 353 op/s 2024-08-16T19:15:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:46 smithi062 ceph-mon[32993]: pgmap v161: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 353 op/s 2024-08-16T19:15:46.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:46 smithi155 ceph-mon[37413]: pgmap v161: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 353 op/s 2024-08-16T19:15:46.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 279888 5.77 MB/sec execute 125 sec latency 21.354 ms 2024-08-16T19:15:47.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 281350 5.78 MB/sec execute 126 sec latency 13.053 ms 2024-08-16T19:15:48.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:48 smithi161 ceph-mon[37576]: pgmap v162: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:48 smithi062 ceph-mon[32993]: pgmap v162: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:48.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:48 smithi155 ceph-mon[37413]: pgmap v162: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:48.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 283535 5.83 MB/sec execute 127 sec latency 29.988 ms 2024-08-16T19:15:49.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 284920 5.84 MB/sec execute 128 sec latency 19.799 ms 2024-08-16T19:15:50.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:50 smithi161 ceph-mon[37576]: pgmap v163: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:50.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:50 smithi062 ceph-mon[32993]: pgmap v163: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:50.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:50 smithi155 ceph-mon[37413]: pgmap v163: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-08-16T19:15:50.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 287141 5.89 MB/sec execute 129 sec latency 12.565 ms 2024-08-16T19:15:51.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 288511 5.90 MB/sec execute 130 sec latency 13.497 ms 2024-08-16T19:15:52.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:52 smithi161 ceph-mon[37576]: pgmap v164: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:52.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:52 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:52.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:52 smithi062 ceph-mon[32993]: pgmap v164: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:52.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:52 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:52.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:52 smithi155 ceph-mon[37413]: pgmap v164: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:52.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:52 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:52.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 290689 5.95 MB/sec execute 131 sec latency 14.311 ms 2024-08-16T19:15:53.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 291823 5.94 MB/sec execute 132 sec latency 40.624 ms 2024-08-16T19:15:54.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:54 smithi161 ceph-mon[37576]: pgmap v165: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-08-16T19:15:54.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:54 smithi062 ceph-mon[32993]: pgmap v165: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-08-16T19:15:54.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:54 smithi155 ceph-mon[37413]: pgmap v165: 129 pgs: 129 active+clean; 184 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-08-16T19:15:54.959 INFO:tasks.workunit.client.0.smithi062.stdout: 1 294244 6.00 MB/sec execute 133 sec latency 13.752 ms 2024-08-16T19:15:55.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 295603 6.01 MB/sec execute 134 sec latency 12.018 ms 2024-08-16T19:15:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:56 smithi161 ceph-mon[37576]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:56.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:56 smithi062 ceph-mon[32993]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:56.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:56 smithi155 ceph-mon[37413]: pgmap v166: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 356 op/s 2024-08-16T19:15:56.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 297800 6.06 MB/sec execute 135 sec latency 13.485 ms 2024-08-16T19:15:57.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:57 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:57.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:57 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:57.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:57 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:15:57.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 299137 6.06 MB/sec execute 136 sec latency 12.804 ms 2024-08-16T19:15:58.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:58 smithi161 ceph-mon[37576]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:58 smithi062 ceph-mon[32993]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:58.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:58 smithi155 ceph-mon[37413]: pgmap v167: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:58.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 301337 6.11 MB/sec execute 137 sec latency 61.011 ms 2024-08-16T19:15:59.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:15:59 smithi161 ceph-mon[37576]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:59.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:15:59 smithi062 ceph-mon[32993]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:59.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:15:59 smithi155 ceph-mon[37413]: pgmap v168: 129 pgs: 129 active+clean; 190 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 252 op/s 2024-08-16T19:15:59.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 302744 6.11 MB/sec execute 138 sec latency 19.064 ms 2024-08-16T19:16:00.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 303811 6.10 MB/sec execute 139 sec latency 60.481 ms 2024-08-16T19:16:01.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:01 smithi062 ceph-mon[32993]: pgmap v169: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 378 op/s 2024-08-16T19:16:01.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:01 smithi155 ceph-mon[37413]: pgmap v169: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 378 op/s 2024-08-16T19:16:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:01 smithi161 ceph-mon[37576]: pgmap v169: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 378 op/s 2024-08-16T19:16:01.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 305440 6.15 MB/sec execute 140 sec latency 57.743 ms 2024-08-16T19:16:02.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 306733 6.14 MB/sec execute 141 sec latency 16.323 ms 2024-08-16T19:16:03.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:03 smithi062 ceph-mon[32993]: pgmap v170: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-08-16T19:16:03.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:03 smithi155 ceph-mon[37413]: pgmap v170: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-08-16T19:16:03.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:03 smithi161 ceph-mon[37576]: pgmap v170: 129 pgs: 129 active+clean; 196 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 252 op/s 2024-08-16T19:16:03.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 308901 6.19 MB/sec execute 142 sec latency 30.832 ms 2024-08-16T19:16:04.906 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:16:04.906 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-08-16T19:16:04.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:16:04.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 310119 6.19 MB/sec execute 143 sec latency 17.216 ms 2024-08-16T19:16:05.184 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:05.559 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:05 smithi062 ceph-mon[32993]: pgmap v171: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 -- 172.21.15.62:0/4238847157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c104460 msgr2=0x7f487c110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 --2- 172.21.15.62:0/4238847157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c104460 0x7f487c110990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4874009cd0 tx=0x7f487402a8e0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 -- 172.21.15.62:0/4238847157 shutdown_connections 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 --2- 172.21.15.62:0/4238847157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c104460 0x7f487c110990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 --2- 172.21.15.62:0/4238847157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c103ac0 0x7f487c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 --2- 172.21.15.62:0/4238847157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c109ac0 0x7f487c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 -- 172.21.15.62:0/4238847157 >> 172.21.15.62:0/4238847157 conn(0x7f487c0ff7c0 msgr2=0x7f487c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:05.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 -- 172.21.15.62:0/4238847157 shutdown_connections 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.639+0000 7f48828fb640 1 -- 172.21.15.62:0/4238847157 wait complete. 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.640+0000 7f48828fb640 1 Processor -- start 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.640+0000 7f48828fb640 1 -- start start 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 0x7f487c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c109ac0 0x7f487c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f487c113f10 con 0x7f487c103ac0 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f487c113d90 con 0x7f487c104460 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f48828fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f487c114090 con 0x7f487c109ac0 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51064/0 (socket says 172.21.15.62:51064) 2024-08-16T19:16:05.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 -- 172.21.15.62:0/3805554132 learned_addr learned my addr 172.21.15.62:0/3805554132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487b7fe640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 0x7f487c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f4880e98640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c109ac0 0x7f487c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 -- 172.21.15.62:0/3805554132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c109ac0 msgr2=0x7f487c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c109ac0 0x7f487c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 -- 172.21.15.62:0/3805554132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 msgr2=0x7f487c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 0x7f487c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 -- 172.21.15.62:0/3805554132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c19f2f0 con 0x7f487c103ac0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487b7fe640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 0x7f487c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.641+0000 7f487bfff640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9f0 tx=0x7f486c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.642+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c00cd80 con 0x7f487c103ac0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.642+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486c010910 con 0x7f487c103ac0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.642+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c19f5e0 con 0x7f487c103ac0 2024-08-16T19:16:05.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.642+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c018990 con 0x7f487c103ac0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.642+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1a6ec0 con 0x7f487c103ac0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.643+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c103f20 con 0x7f487c103ac0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.644+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f486c010ac0 con 0x7f487c103ac0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.644+0000 7f48797fa640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 0x7f48640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.645+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f486c0cfa70 con 0x7f487c103ac0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.645+0000 7f487b7fe640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 0x7f48640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.645+0000 7f487b7fe640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 0x7f48640a51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f487c101cf0 tx=0x7f4870005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:05.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.649+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f486c09bcb0 con 0x7f487c103ac0 2024-08-16T19:16:05.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:05 smithi155 ceph-mon[37413]: pgmap v171: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-08-16T19:16:05.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.869+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f487c1a7150 con 0x7f487c103ac0 2024-08-16T19:16:05.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.870+0000 7f48797fa640 1 -- 172.21.15.62:0/3805554132 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+12006 (secure 0 0 0) 0x7f486c09ffb0 con 0x7f487c103ac0 2024-08-16T19:16:05.871 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:16:05.871 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"btime":"2024-08-16T19:15:56:103555+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:15:55.845588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:16:05.871 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 msgr2=0x7f48640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 0x7f48640a51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f487c101cf0 tx=0x7f4870005f90 comp rx=0 tx=0).stop 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 msgr2=0x7f487c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9f0 tx=0x7f486c00eeb0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 shutdown_connections 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f48640a2cf0 0x7f48640a51b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f487c109ac0 0x7f487c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f487c104460 0x7f487c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 --2- 172.21.15.62:0/3805554132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f487c103ac0 0x7f487c19a340 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 >> 172.21.15.62:0/3805554132 conn(0x7f487c0ff7c0 msgr2=0x7f487c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 shutdown_connections 2024-08-16T19:16:05.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:05.872+0000 7f48828fb640 1 -- 172.21.15.62:0/3805554132 wait complete. 2024-08-16T19:16:05.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:05 smithi161 ceph-mon[37576]: pgmap v171: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 398 op/s 2024-08-16T19:16:05.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 312380 6.24 MB/sec execute 144 sec latency 58.749 ms 2024-08-16T19:16:06.482 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-08-16T19:16:06.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-08-16T19:16:06.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:06 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3805554132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:06.757 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:06.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:06 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3805554132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:06.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:06 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3805554132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:06.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313580 6.23 MB/sec execute 145 sec latency 55.571 ms 2024-08-16T19:16:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 -- 172.21.15.62:0/3414172460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 msgr2=0x7f4ad010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 --2- 172.21.15.62:0/3414172460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad010d770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8009ca0 tx=0x7f4ac802a950 comp rx=0 tx=0).stop 2024-08-16T19:16:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 -- 172.21.15.62:0/3414172460 shutdown_connections 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 --2- 172.21.15.62:0/3414172460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad010d770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 --2- 172.21.15.62:0/3414172460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad01022a0 0x7f4ad01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 --2- 172.21.15.62:0/3414172460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad0100b50 0x7f4ad0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 -- 172.21.15.62:0/3414172460 >> 172.21.15.62:0/3414172460 conn(0x7f4ad00fc500 msgr2=0x7f4ad00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 -- 172.21.15.62:0/3414172460 shutdown_connections 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.213+0000 7f4ad624c640 1 -- 172.21.15.62:0/3414172460 wait complete. 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 Processor -- start 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 -- start start 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad0100b50 0x7f4ad0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 0x7f4ad0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0110cb0 con 0x7f4ad0102c20 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0110b30 con 0x7f4ad01022a0 2024-08-16T19:16:07.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4ad624c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0110e30 con 0x7f4ad0100b50 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4acffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad0100b50 0x7f4ad0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4acffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51090/0 (socket says 172.21.15.62:51090) 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4acffff640 1 -- 172.21.15.62:0/2912096600 learned_addr learned my addr 172.21.15.62:0/2912096600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.214+0000 7f4aceffd640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 0x7f4ad0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 -- 172.21.15.62:0/2912096600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad0100b50 msgr2=0x7f4ad0198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad0100b50 0x7f4ad0198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 -- 172.21.15.62:0/2912096600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 msgr2=0x7f4ad0199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 0x7f4ad0199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 -- 172.21.15.62:0/2912096600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01a45a0 con 0x7f4ad0102c20 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4aceffd640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 0x7f4ad0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4acffff640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8004af0 tx=0x7f4ac802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4accff9640 1 -- 172.21.15.62:0/2912096600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8034040 con 0x7f4ad0102c20 2024-08-16T19:16:07.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01a4770 con 0x7f4ad0102c20 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4accff9640 1 -- 172.21.15.62:0/2912096600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8030920 con 0x7f4ad0102c20 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4accff9640 1 -- 172.21.15.62:0/2912096600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004630 con 0x7f4ad0102c20 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.215+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a4c20 con 0x7f4ad0102c20 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.217+0000 7f4accff9640 1 -- 172.21.15.62:0/2912096600 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4ac8037020 con 0x7f4ad0102c20 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 Processor -- start 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 -- start start 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4accff9640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 0x7f4ab80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 0x56268d34c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d35dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56268d2b1810 con 0x56268d34caa0 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56268d258c70 con 0x56268d34c180 2024-08-16T19:16:07.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4ad864a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56268d2ad7d0 con 0x56268d35b8b0 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4accff9640 1 -- 172.21.15.62:0/2912096600 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f4ac80f03c0 con 0x7f4ad0102c20 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4aceffd640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 0x7f4ab80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.218+0000 7f4aceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 0x56268d34c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4aceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51094/0 (socket says 172.21.15.62:51094) 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4aceffd640 1 -- 172.21.15.62:0/3458417359 learned_addr learned my addr 172.21.15.62:0/3458417359 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acffff640 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d35dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 -- 172.21.15.62:0/3458417359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 msgr2=0x56268d35dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d35dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 -- 172.21.15.62:0/3458417359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 msgr2=0x56268d35b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 -- 172.21.15.62:0/3458417359 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268d023ba0 con 0x56268d34c180 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4aceffd640 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4aceffd640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 0x7f4ab80a5230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4011d10 tx=0x7f4ac4009f50 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 0x56268d34c560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009c20 tx=0x7f4ac0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6981e7c5f95f219 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4ab5ffb640 1 -- 172.21.15.62:0/3458417359 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0004050 con 0x56268d34c180 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4ab5ffb640 1 -- 172.21.15.62:0/3458417359 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0005500 con 0x56268d34c180 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.219+0000 7f4ab5ffb640 1 -- 172.21.15.62:0/3458417359 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0004d90 con 0x56268d34c180 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 -- 172.21.15.62:0/3458417359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 msgr2=0x56268d34c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 0x56268d34c560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009c20 tx=0x7f4ac0011810 comp rx=0 tx=0).stop 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 -- 172.21.15.62:0/3458417359 shutdown_connections 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d35dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d35b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 --2- 172.21.15.62:0/3458417359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34c180 0x56268d34c560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 -- 172.21.15.62:0/3458417359 >> 172.21.15.62:0/3458417359 conn(0x56268d348850 msgr2=0x56268d348c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 -- 172.21.15.62:0/3458417359 shutdown_connections 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.220+0000 7f4ad864a740 1 -- 172.21.15.62:0/3458417359 wait complete. 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 -- start start 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 Processor -- start 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34c180 0x56268d358750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d3591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56268d35e480 con 0x56268d34caa0 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x56268d26c9a0 con 0x56268d35b8b0 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56268d35e600 con 0x56268d34c180 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4ad864a740 10 client.0 _ll_register_callbacks cb 0x56268d25ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4aceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34c180 0x56268d358750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4aceffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51098/0 (socket says 172.21.15.62:51098) 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4aceffd640 1 -- 172.21.15.62:0/689085109 learned_addr learned my addr 172.21.15.62:0/689085109 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.221+0000 7f4acffff640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d3591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 -- 172.21.15.62:0/689085109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34c180 msgr2=0x56268d358750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34c180 0x56268d358750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 -- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 msgr2=0x56268d3591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d3591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 -- 172.21.15.62:0/689085109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268d278220 con 0x56268d34caa0 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aceffd640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ac401e950 tx=0x7f4ac4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44bfb28df800f84 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4004020 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac0005500 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f4ac40041c0 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4ac4032050 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4ad864a740 10 client.15780 fetch_fsmap learned FSMap version 38 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56268d36a170 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.222+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0005500 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8410+0+0 (secure 0 0 0) 0x7f4ac4003610 con 0x56268d34caa0 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 10 client.15780 fetch_fsmap finished waiting for FSMap version 38 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 10 client.15780 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 20 client.15780 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d36b690 0x56268d36da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 4 client.15780 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:16:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56268d2bef40 con 0x56268d36b690 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d36b690 0x56268d36da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d36b690 0x56268d36da80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d7e0c0366b8734 server_cookie=fd468d9ed0e122e5 in_seq=0 out_seq=0 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.223+0000 7f4aaeffd640 10 client.15780 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4aaeffd640 1 -- 172.21.15.62:0/689085109 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56268d2bef40 con 0x56268d36b690 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4aaeffd640 10 client.15780 handle_command_reply: tid=0 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 client.15780 shutdown 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 msgr2=0x56268d358c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ac401e950 tx=0x7f4ac4005f90 comp rx=0 tx=0).stop 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 shutdown_connections 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d36b690 0x56268d36da80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d3591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34caa0 0x56268d358c90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 --2- 172.21.15.62:0/689085109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34c180 0x56268d358750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 >> 172.21.15.62:0/689085109 conn(0x56268d348850 msgr2=0x56268d347b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.224+0000 7f4aaeffd640 0 client.15780 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.225+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 shutdown_connections 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.225+0000 7f4ad864a740 1 -- 172.21.15.62:0/689085109 wait complete. 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.225+0000 7f4ad864a740 20 client.15780 trim_cache size 0 max 16384 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 Processor -- start 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 -- start start 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34caa0 0x56268d373390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d3738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01aca50 con 0x56268d34c180 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ae710 con 0x56268d34caa0 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4ad864a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ad120 con 0x56268d35b8b0 2024-08-16T19:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51108/0 (socket says 172.21.15.62:51108) 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1012892355 learned_addr learned my addr 172.21.15.62:0/1012892355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4aceffd640 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34caa0 0x56268d373390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acffff640 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d3738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1012892355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 msgr2=0x56268d3738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d3738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1012892355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34caa0 msgr2=0x56268d373390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34caa0 0x56268d373390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1012892355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268d36ddf0 con 0x56268d34c180 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.243+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000c00 tx=0x7f4ac0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f9d0a62dfdfc80 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ab57fa640 1 -- 172.21.15.62:0/1012892355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0033070 con 0x56268d34c180 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ab57fa640 1 -- 172.21.15.62:0/1012892355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac00035f0 con 0x56268d34c180 2024-08-16T19:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ab57fa640 1 -- 172.21.15.62:0/1012892355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac002e640 con 0x56268d34c180 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 -- 172.21.15.62:0/1012892355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 msgr2=0x56268d347a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000c00 tx=0x7f4ac0009680 comp rx=0 tx=0).stop 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 -- 172.21.15.62:0/1012892355 shutdown_connections 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d35b8b0 0x56268d3738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d34caa0 0x56268d373390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1012892355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d347a20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 -- 172.21.15.62:0/1012892355 >> 172.21.15.62:0/1012892355 conn(0x56268d348850 msgr2=0x56268d347310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.244+0000 7f4ad864a740 1 -- 172.21.15.62:0/1012892355 shutdown_connections 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 -- 172.21.15.62:0/1012892355 wait complete. 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 -- start start 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 Processor -- start 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 0x56268d35fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d360000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01adf80 con 0x56268d34c180 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ad7f0 con 0x56268d35b8b0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01aa9d0 con 0x56268d34caa0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4ad864a740 10 client.0 _ll_register_callbacks cb 0x7f4ac4011740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.245+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51116/0 (socket says 172.21.15.62:51116) 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1058828957 learned_addr learned my addr 172.21.15.62:0/1058828957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aceffd640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 0x56268d35fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acffff640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d360000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1058828957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 msgr2=0x56268d35fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 0x56268d35fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 msgr2=0x56268d360000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:07.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d360000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 -- 172.21.15.62:0/1058828957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268d3d5360 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aceffd640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 0x56268d35fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acffff640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d360000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4acf7fe640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0011e90 tx=0x7f4ac0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd77af9f838e32c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac003b020 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f4ac0003bb0 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac00035f0 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4ac0033050 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4ad864a740 10 client.15786 fetch_fsmap learned FSMap version 38 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.246+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56268d3d65c0 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac00035f0 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) ==== 8410+0+0 (secure 0 0 0) 0x7f4ac002e910 con 0x56268d34c180 2024-08-16T19:16:07.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 10 client.15786 fetch_fsmap finished waiting for FSMap version 38 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 10 client.15786 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 20 client.15786 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d3d6bd0 0x56268d3d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 4 client.15786 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x56268d2b9290 con 0x56268d3d6bd0 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4aceffd640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d3d6bd0 0x56268d3d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4aceffd640 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d3d6bd0 0x56268d3d8ff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebef8513c490171 server_cookie=dfaa5e27d571257a in_seq=0 out_seq=0 2024-08-16T19:16:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.247+0000 7f4aaeffd640 10 client.15786 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4aaeffd640 1 -- 172.21.15.62:0/1058828957 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56268d2b9290 con 0x56268d3d6bd0 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4aaeffd640 10 client.15786 handle_command_reply: tid=0 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 client.15786 shutdown 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 msgr2=0x56268d35f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0011e90 tx=0x7f4ac0009b20 comp rx=0 tx=0).stop 2024-08-16T19:16:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 shutdown_connections 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56268d3d6bd0 0x56268d3d8ff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56268d35b8b0 0x56268d360000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56268d34caa0 0x56268d35fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4aaeffd640 0 client.15786 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 --2- 172.21.15.62:0/1058828957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56268d34c180 0x56268d35f580 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.322+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 >> 172.21.15.62:0/1058828957 conn(0x56268d348850 msgr2=0x56268d347b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 shutdown_connections 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad864a740 1 -- 172.21.15.62:0/1058828957 wait complete. 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad864a740 20 client.15786 trim_cache size 0 max 16384 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:16:07.336 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 38, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 22, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "d8bca99": { 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 22, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:16:07.337 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 141.1, 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:07.338 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 msgr2=0x7f4ab80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 0x7f4ab80a5230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4011d10 tx=0x7f4ac4009f50 comp rx=0 tx=0).stop 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 msgr2=0x7f4ad01a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8004af0 tx=0x7f4ac802f3a0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 shutdown_connections 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f4ab80a2d70 0x7f4ab80a5230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad0102c20 0x7f4ad01a4060 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ad01022a0 0x7f4ad0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 --2- 172.21.15.62:0/2912096600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ad0100b50 0x7f4ad0198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.323+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 >> 172.21.15.62:0/2912096600 conn(0x7f4ad00fc500 msgr2=0x7f4ad0106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.324+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 shutdown_connections 2024-08-16T19:16:07.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:07.324+0000 7f4ad624c640 1 -- 172.21.15.62:0/2912096600 wait complete. 2024-08-16T19:16:07.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:07 smithi062 ceph-mon[32993]: pgmap v172: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:07 smithi155 ceph-mon[37413]: pgmap v172: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:07 smithi161 ceph-mon[37576]: pgmap v172: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:07.932 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd8bca99', quiesce took 3.0 seconds. Will release after: 18.05 2024-08-16T19:16:07.960 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 6.19 MB/sec execute 146 sec latency 668.854 ms 2024-08-16T19:16:08.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 6.15 MB/sec execute 147 sec latency 1668.982 ms 2024-08-16T19:16:09.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:09 smithi062 ceph-mon[32993]: pgmap v173: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:09 smithi155 ceph-mon[37413]: pgmap v173: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:09 smithi161 ceph-mon[37576]: pgmap v173: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-08-16T19:16:09.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 6.11 MB/sec execute 148 sec latency 2669.416 ms 2024-08-16T19:16:10.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 6.07 MB/sec execute 149 sec latency 3669.512 ms 2024-08-16T19:16:11.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:11 smithi062 ceph-mon[32993]: pgmap v174: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-08-16T19:16:11.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:11 smithi155 ceph-mon[37413]: pgmap v174: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-08-16T19:16:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:11 smithi161 ceph-mon[37576]: pgmap v174: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-08-16T19:16:11.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 6.03 MB/sec execute 150 sec latency 4669.628 ms 2024-08-16T19:16:12.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.99 MB/sec execute 151 sec latency 5669.708 ms 2024-08-16T19:16:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:13 smithi062 ceph-mon[32993]: pgmap v175: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-08-16T19:16:13.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:13 smithi155 ceph-mon[37413]: pgmap v175: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-08-16T19:16:13.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:13 smithi161 ceph-mon[37576]: pgmap v175: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 228 op/s 2024-08-16T19:16:13.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.95 MB/sec execute 152 sec latency 6669.788 ms 2024-08-16T19:16:14.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.91 MB/sec execute 153 sec latency 7669.876 ms 2024-08-16T19:16:15.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:15 smithi062 ceph-mon[32993]: pgmap v176: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 259 op/s 2024-08-16T19:16:15.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:15 smithi155 ceph-mon[37413]: pgmap v176: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 259 op/s 2024-08-16T19:16:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:15 smithi161 ceph-mon[37576]: pgmap v176: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 259 op/s 2024-08-16T19:16:15.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.87 MB/sec execute 154 sec latency 8669.933 ms 2024-08-16T19:16:16.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.83 MB/sec execute 155 sec latency 9670.039 ms 2024-08-16T19:16:17.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:17 smithi062 ceph-mon[32993]: pgmap v177: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:17.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:17 smithi155 ceph-mon[37413]: pgmap v177: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:17.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:17 smithi161 ceph-mon[37576]: pgmap v177: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:17.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.79 MB/sec execute 156 sec latency 10670.159 ms 2024-08-16T19:16:18.961 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.76 MB/sec execute 157 sec latency 11670.213 ms 2024-08-16T19:16:19.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:19 smithi062 ceph-mon[32993]: pgmap v178: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:19.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:19 smithi155 ceph-mon[37413]: pgmap v178: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:19.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:19 smithi161 ceph-mon[37576]: pgmap v178: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 113 op/s 2024-08-16T19:16:19.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.72 MB/sec execute 158 sec latency 12670.330 ms 2024-08-16T19:16:20.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.68 MB/sec execute 159 sec latency 13670.432 ms 2024-08-16T19:16:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:21 smithi062 ceph-mon[32993]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-08-16T19:16:21.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:21 smithi155 ceph-mon[37413]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-08-16T19:16:21.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:21 smithi161 ceph-mon[37576]: pgmap v179: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-08-16T19:16:21.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.65 MB/sec execute 160 sec latency 14670.562 ms 2024-08-16T19:16:22.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.61 MB/sec execute 161 sec latency 15670.685 ms 2024-08-16T19:16:23.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:23 smithi062 ceph-mon[32993]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-08-16T19:16:23.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:23 smithi155 ceph-mon[37413]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-08-16T19:16:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:23 smithi161 ceph-mon[37576]: pgmap v180: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-08-16T19:16:23.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.58 MB/sec execute 162 sec latency 16670.811 ms 2024-08-16T19:16:24.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.55 MB/sec execute 163 sec latency 17670.887 ms 2024-08-16T19:16:25.826 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:25 smithi062 ceph-mon[32993]: pgmap v181: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-08-16T19:16:25.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:25 smithi155 ceph-mon[37413]: pgmap v181: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-08-16T19:16:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:25 smithi161 ceph-mon[37576]: pgmap v181: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-08-16T19:16:25.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.51 MB/sec execute 164 sec latency 18670.947 ms 2024-08-16T19:16:25.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:16:26.267 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 -- 172.21.15.62:0/628148382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04107fd0 msgr2=0x7f9c0410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 --2- 172.21.15.62:0/628148382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04107fd0 0x7f9c0410e970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009cd0 tx=0x7f9bfc02a8e0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 -- 172.21.15.62:0/628148382 shutdown_connections 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 --2- 172.21.15.62:0/628148382 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04107fd0 0x7f9c0410e970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 --2- 172.21.15.62:0/628148382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04103410 0x7f9c041078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 --2- 172.21.15.62:0/628148382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04102a60 0x7f9c04102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 -- 172.21.15.62:0/628148382 >> 172.21.15.62:0/628148382 conn(0x7f9c040fc500 msgr2=0x7f9c040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:26.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 -- 172.21.15.62:0/628148382 shutdown_connections 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.741+0000 7f9c08adc640 1 -- 172.21.15.62:0/628148382 wait complete. 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 Processor -- start 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 -- start start 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 0x7f9c04100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04107fd0 0x7f9c04101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04111dd0 con 0x7f9c04102a60 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04111c50 con 0x7f9c04103410 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c08adc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04111f50 con 0x7f9c04107fd0 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c0259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c0259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33726/0 (socket says 172.21.15.62:33726) 2024-08-16T19:16:26.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c0259c640 1 -- 172.21.15.62:0/2171848115 learned_addr learned my addr 172.21.15.62:0/2171848115 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9c02d9d640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04107fd0 0x7f9c04101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.742+0000 7f9bfbfff640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 0x7f9c04100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 -- 172.21.15.62:0/2171848115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04107fd0 msgr2=0x7f9c04101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04107fd0 0x7f9c04101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 -- 172.21.15.62:0/2171848115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 msgr2=0x7f9c04100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:26.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 0x7f9c04100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 -- 172.21.15.62:0/2171848115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c041a66c0 con 0x7f9c04102a60 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9bfbfff640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 0x7f9c04100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c0259c640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c920 tx=0x7f9bf400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4007c70 con 0x7f9c04102a60 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4007e10 con 0x7f9c04102a60 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041a69b0 con 0x7f9c04102a60 2024-08-16T19:16:26.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4018890 con 0x7f9c04102a60 2024-08-16T19:16:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.743+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041a6f40 con 0x7f9c04102a60 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.744+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c04102e40 con 0x7f9c04102a60 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.746+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9bf4018a30 con 0x7f9c04102a60 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.747+0000 7f9bfb7fe640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 0x7f9bf00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.747+0000 7f9bfbfff640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 0x7f9bf00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.747+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f9bf40cfc40 con 0x7f9c04102a60 2024-08-16T19:16:26.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.747+0000 7f9bfbfff640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 0x7f9bf00a5230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bec005fd0 tx=0x7f9bec005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:26.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.750+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9bf4014030 con 0x7f9c04102a60 2024-08-16T19:16:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.961+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c04101bf0 con 0x7f9c04102a60 2024-08-16T19:16:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.962+0000 7f9bfb7fe640 1 -- 172.21.15.62:0/2171848115 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12000 (secure 0 0 0) 0x7f9bf400de20 con 0x7f9c04102a60 2024-08-16T19:16:26.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:16:26.962 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-08-16T19:16:26:496652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:16:25.577802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:16:26.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.48 MB/sec execute 165 sec latency 19670.977 ms 2024-08-16T19:16:26.963 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-08-16T19:16:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.963+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 msgr2=0x7f9bf00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.963+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 0x7f9bf00a5230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bec005fd0 tx=0x7f9bec005850 comp rx=0 tx=0).stop 2024-08-16T19:16:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.963+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 msgr2=0x7f9c041006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.963+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c920 tx=0x7f9bf400cde0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 shutdown_connections 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9bf00a2d70 0x7f9bf00a5230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c04107fd0 0x7f9c04101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c04103410 0x7f9c04100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 --2- 172.21.15.62:0/2171848115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c04102a60 0x7f9c041006c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 >> 172.21.15.62:0/2171848115 conn(0x7f9c040fc500 msgr2=0x7f9c041060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 shutdown_connections 2024-08-16T19:16:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:26.964+0000 7f9c08adc640 1 -- 172.21.15.62:0/2171848115 wait complete. 2024-08-16T19:16:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:27 smithi062 ceph-mon[32993]: pgmap v182: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:27 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:27.747 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:27 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2171848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:27 smithi155 ceph-mon[37413]: pgmap v182: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:27 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:27 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2171848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:27 smithi161 ceph-mon[37576]: pgmap v182: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:27 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:27 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2171848115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:27.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 313850 5.45 MB/sec execute 166 sec latency 20671.059 ms 2024-08-16T19:16:28.000 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d8bca99 --release --await' 2024-08-16T19:16:28.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d8bca99 --release --await 2024-08-16T19:16:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 -- 172.21.15.62:0/1072493342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 msgr2=0x7f5a1c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 --2- 172.21.15.62:0/1072493342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c102e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a1802a480 comp rx=0 tx=0).stop 2024-08-16T19:16:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 -- 172.21.15.62:0/1072493342 shutdown_connections 2024-08-16T19:16:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 --2- 172.21.15.62:0/1072493342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 --2- 172.21.15.62:0/1072493342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c103390 0x7f5a1c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 --2- 172.21.15.62:0/1072493342 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c102e50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 -- 172.21.15.62:0/1072493342 >> 172.21.15.62:0/1072493342 conn(0x7f5a1c0fc4d0 msgr2=0x7f5a1c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 -- 172.21.15.62:0/1072493342 shutdown_connections 2024-08-16T19:16:28.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.733+0000 7f5a24508640 1 -- 172.21.15.62:0/1072493342 wait complete. 2024-08-16T19:16:28.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 Processor -- start 2024-08-16T19:16:28.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 -- start start 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c103390 0x7f5a1c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c111ca0 con 0x7f5a1c102a70 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c111b20 con 0x7f5a1c10bea0 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a24508640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c111e20 con 0x7f5a1c103390 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33736/0 (socket says 172.21.15.62:33736) 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.734+0000 7f5a222a4640 1 -- 172.21.15.62:0/636618551 learned_addr learned my addr 172.21.15.62:0/636618551 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c103390 0x7f5a1c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 -- 172.21.15.62:0/636618551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c103390 msgr2=0x7f5a1c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c103390 0x7f5a1c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 -- 172.21.15.62:0/636618551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 msgr2=0x7f5a1c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 -- 172.21.15.62:0/636618551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c19d110 con 0x7f5a1c102a70 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a222a4640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a137fe640 1 -- 172.21.15.62:0/636618551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1803f030 con 0x7f5a1c102a70 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c19d3a0 con 0x7f5a1c102a70 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a137fe640 1 -- 172.21.15.62:0/636618551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18003ce0 con 0x7f5a1c102a70 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a137fe640 1 -- 172.21.15.62:0/636618551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18035070 con 0x7f5a1c102a70 2024-08-16T19:16:28.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.735+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1a4ce0 con 0x7f5a1c102a70 2024-08-16T19:16:28.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a137fe640 1 -- 172.21.15.62:0/636618551 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a18042030 con 0x7f5a1c102a70 2024-08-16T19:16:28.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 Processor -- start 2024-08-16T19:16:28.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 -- start start 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafbec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafcda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620daf24c60 con 0x5620dafbe850 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620daebcb90 con 0x5620dafcdf80 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.738+0000 7f5a26906740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620daf20c20 con 0x5620dafbf170 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a137fe640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a22aa5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafbec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafcda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a22aa5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafbec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33744/0 (socket says 172.21.15.62:33744) 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a22aa5640 1 -- 172.21.15.62:0/1450643901 learned_addr learned my addr 172.21.15.62:0/1450643901 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a137fe640 1 -- 172.21.15.62:0/636618551 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f5a180efea0 con 0x7f5a1c102a70 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 -- 172.21.15.62:0/1450643901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 msgr2=0x5620dafd0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafd0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 -- 172.21.15.62:0/1450643901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 msgr2=0x5620dafbec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafbec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 -- 172.21.15.62:0/1450643901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dac34580 con 0x5620dafbf170 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 0x7f5a080a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a140092f0 tx=0x7f5a14009990 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.739+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafcda40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ed260 tx=0x7f5a180b1580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c82cb3cabd05c20 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a10ff9640 1 -- 172.21.15.62:0/1450643901 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18049070 con 0x5620dafbf170 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a10ff9640 1 -- 172.21.15.62:0/1450643901 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1803b580 con 0x5620dafbf170 2024-08-16T19:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a10ff9640 1 -- 172.21.15.62:0/1450643901 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18049e40 con 0x5620dafbf170 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 -- 172.21.15.62:0/1450643901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 msgr2=0x5620dafcda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafcda40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a180ed260 tx=0x7f5a180b1580 comp rx=0 tx=0).stop 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 -- 172.21.15.62:0/1450643901 shutdown_connections 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafcda40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 --2- 172.21.15.62:0/1450643901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafbec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.740+0000 7f5a26906740 1 -- 172.21.15.62:0/1450643901 >> 172.21.15.62:0/1450643901 conn(0x5620dafbaf20 msgr2=0x5620dafbb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- 172.21.15.62:0/1450643901 shutdown_connections 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- 172.21.15.62:0/1450643901 wait complete. 2024-08-16T19:16:28.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- start start 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 Processor -- start 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 0x5620dafcaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafcb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5620dafd0910 con 0x5620dafbf170 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5620daec27c0 con 0x5620dafcdf80 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5620dafd0a90 con 0x5620dafbe850 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a26906740 10 client.0 _ll_register_callbacks cb 0x5620daecbf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a21aa3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 0x5620dafcaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a21aa3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33756/0 (socket says 172.21.15.62:33756) 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.741+0000 7f5a21aa3640 1 -- 172.21.15.62:0/1442535531 learned_addr learned my addr 172.21.15.62:0/1442535531 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:28.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafcb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 -- 172.21.15.62:0/1442535531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 msgr2=0x5620dafcaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 0x5620dafcaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 -- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 msgr2=0x5620dafcb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafcb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 -- 172.21.15.62:0/1442535531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dacfe960 con 0x5620dafbf170 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 0x5620dafcaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00bf70 tx=0x7f5a0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992fcf82bf046d8b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c004020 con 0x5620dafbf170 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f5a0c0041c0 con 0x5620dafbf170 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620daf321d0 con 0x5620dafbf170 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c024050 con 0x5620dafbf170 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a26906740 10 client.15804 fetch_fsmap learned FSMap version 39 2024-08-16T19:16:28.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5620dafdc990 con 0x5620dafbf170 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.742+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5620daf321d0 con 0x5620dafbf170 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8398+0+0 (secure 0 0 0) 0x7f5a0c0038c0 con 0x5620dafbf170 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 10 client.15804 fetch_fsmap finished waiting for FSMap version 39 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 10 client.15804 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 20 client.15804 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620dafddea0 0x5620dafe0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 4 client.15804 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620daf2c6e0 con 0x5620dafddea0 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.743+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620dafddea0 0x5620dafe0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a222a4640 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620dafddea0 0x5620dafe0360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fdffeb97632a5c server_cookie=69dbca5f921912ef in_seq=0 out_seq=0 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a05ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a05ffb640 1 -- 172.21.15.62:0/1442535531 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620daf2c6e0 con 0x5620dafddea0 2024-08-16T19:16:28.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a05ffb640 10 client.15804 handle_command_reply: tid=0 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 client.15804 shutdown 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 msgr2=0x5620dafcb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00bf70 tx=0x7f5a0c011810 comp rx=0 tx=0).stop 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 shutdown_connections 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620dafddea0 0x5620dafe0360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafcb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbf170 0x5620dafcb480 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 --2- 172.21.15.62:0/1442535531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbe850 0x5620dafcaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 >> 172.21.15.62:0/1442535531 conn(0x5620dafbaf20 msgr2=0x5620dafc0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.744+0000 7f5a05ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.745+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 shutdown_connections 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.745+0000 7f5a26906740 1 -- 172.21.15.62:0/1442535531 wait complete. 2024-08-16T19:16:28.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.745+0000 7f5a26906740 20 client.15804 trim_cache size 0 max 16384 2024-08-16T19:16:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 Processor -- start 2024-08-16T19:16:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 -- start start 2024-08-16T19:16:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafe6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafc81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1acc20 con 0x5620dafbe850 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1ae850 con 0x5620dafcdf80 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a26906740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1ad2f0 con 0x5620dafbf170 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33762/0 (socket says 172.21.15.62:33762) 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 -- 172.21.15.62:0/4082391616 learned_addr learned my addr 172.21.15.62:0/4082391616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafc81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafe6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 -- 172.21.15.62:0/4082391616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 msgr2=0x5620dafe6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafe6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 -- 172.21.15.62:0/4082391616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 msgr2=0x5620dafc81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafc81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.762+0000 7f5a222a4640 1 -- 172.21.15.62:0/4082391616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dafba9c0 con 0x5620dafbe850 2024-08-16T19:16:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a222a4640 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a1803b510 tx=0x7f5a18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa17b0586a2664e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a07fff640 1 -- 172.21.15.62:0/4082391616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180b6070 con 0x5620dafbe850 2024-08-16T19:16:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a07fff640 1 -- 172.21.15.62:0/4082391616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a180bc7f0 con 0x5620dafbe850 2024-08-16T19:16:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a07fff640 1 -- 172.21.15.62:0/4082391616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18049040 con 0x5620dafbe850 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 -- 172.21.15.62:0/4082391616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 msgr2=0x5620dafe5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a1803b510 tx=0x7f5a18004270 comp rx=0 tx=0).stop 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 -- 172.21.15.62:0/4082391616 shutdown_connections 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620dafc81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafe6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 --2- 172.21.15.62:0/4082391616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafe5c30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.763+0000 7f5a26906740 1 -- 172.21.15.62:0/4082391616 >> 172.21.15.62:0/4082391616 conn(0x5620dafbaf20 msgr2=0x5620dafca030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- 172.21.15.62:0/4082391616 shutdown_connections 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- 172.21.15.62:0/4082391616 wait complete. 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- start start 2024-08-16T19:16:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 Processor -- start 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafd15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620db046bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1ae0c0 con 0x5620dafbe850 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1ad9c0 con 0x5620dafcdf80 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c1aabc0 con 0x5620dafbf170 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a26906740 10 client.0 _ll_register_callbacks cb 0x7f5a0c00ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a222a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33768/0 (socket says 172.21.15.62:33768) 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a222a4640 1 -- 172.21.15.62:0/3071935753 learned_addr learned my addr 172.21.15.62:0/3071935753 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.764+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafd15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a22aa5640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620db046bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 -- 172.21.15.62:0/3071935753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 msgr2=0x5620dafd15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafd15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 -- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 msgr2=0x5620db046bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620db046bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 -- 172.21.15.62:0/3071935753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620db047130 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a222a4640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005940 tx=0x7f5a180c4e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04ceb4792acf27 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180b6070 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f5a180c0180 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a180bc7f0 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a18049040 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a26906740 10 client.15810 fetch_fsmap learned FSMap version 39 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5620db048350 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.765+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a180bc7f0 con 0x5620dafbe850 2024-08-16T19:16:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 39) ==== 8398+0+0 (secure 0 0 0) 0x7f5a180c48d0 con 0x5620dafbe850 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 10 client.15810 fetch_fsmap finished waiting for FSMap version 39 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 10 client.15810 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 20 client.15810 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620db0485f0 0x5620db04a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 4 client.15810 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d8bca99", "release": true, "await": true}] 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d8bca99", "release": true, "await": true}) -- 0x5620daea95a0 con 0x5620db0485f0 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620db0485f0 0x5620db04a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a05ffb640 10 client.15810 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:28.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.766+0000 7f5a21aa3640 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620db0485f0 0x5620db04a9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260a5cdfcc85e16c server_cookie=635324074179ad26 in_seq=0 out_seq=0 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a05ffb640 1 -- 172.21.15.62:0/3071935753 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5620daea95a0 con 0x5620db0485f0 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a05ffb640 10 client.15810 handle_command_reply: tid=0 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 client.15810 shutdown 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 msgr2=0x5620dafd1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005940 tx=0x7f5a180c4e50 comp rx=0 tx=0).stop 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 shutdown_connections 2024-08-16T19:16:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5620db0485f0 0x5620db04a9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620dafcdf80 0x5620db046bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5620dafbf170 0x5620dafd15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 --2- 172.21.15.62:0/3071935753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5620dafbe850 0x5620dafd1060 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 >> 172.21.15.62:0/3071935753 conn(0x5620dafbaf20 msgr2=0x5620dafb9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a05ffb640 0 client.15810 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 shutdown_connections 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 1 -- 172.21.15.62:0/3071935753 wait complete. 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.774+0000 7f5a26906740 20 client.15810 trim_cache size 0 max 16384 2024-08-16T19:16:28.776 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 39, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 24, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "d8bca99": { 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 24, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 141.1, 2024-08-16T19:16:28.777 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:28.778 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 msgr2=0x7f5a080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 0x7f5a080a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a140092f0 tx=0x7f5a14009990 comp rx=0 tx=0).stop 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 msgr2=0x7f5a1c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a180042f0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 shutdown_connections 2024-08-16T19:16:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f5a080a2cf0 0x7f5a080a51b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c10bea0 0x7f5a1c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a1c103390 0x7f5a1c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 --2- 172.21.15.62:0/636618551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a1c102a70 0x7f5a1c198160 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 >> 172.21.15.62:0/636618551 conn(0x7f5a1c0fc4d0 msgr2=0x7f5a1c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 shutdown_connections 2024-08-16T19:16:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:28.775+0000 7f5a24508640 1 -- 172.21.15.62:0/636618551 wait complete. 2024-08-16T19:16:28.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 314545 5.42 MB/sec execute 167 sec latency 21486.551 ms 2024-08-16T19:16:29.442 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd8bca99', total seconds elapsed: 24.5 2024-08-16T19:16:29.442 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-08-16T19:16:29.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:16:29.727 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:29 smithi062 ceph-mon[32993]: pgmap v183: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:29.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:29 smithi155 ceph-mon[37413]: pgmap v183: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:29 smithi161 ceph-mon[37576]: pgmap v183: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:16:29.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 316149 5.46 MB/sec execute 168 sec latency 27.905 ms 2024-08-16T19:16:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 -- 172.21.15.62:0/3164861502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281078c0 msgr2=0x7ff228107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 --2- 172.21.15.62:0/3164861502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281078c0 0x7ff228107ca0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff218009cd0 tx=0x7ff21802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 -- 172.21.15.62:0/3164861502 shutdown_connections 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 --2- 172.21.15.62:0/3164861502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff228102260 0x7ff22810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 --2- 172.21.15.62:0/3164861502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2281018c0 0x7ff228101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 --2- 172.21.15.62:0/3164861502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281078c0 0x7ff228107ca0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.134+0000 7ff22da60640 1 -- 172.21.15.62:0/3164861502 >> 172.21.15.62:0/3164861502 conn(0x7ff2280fd5e0 msgr2=0x7ff2280ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.135+0000 7ff22da60640 1 -- 172.21.15.62:0/3164861502 shutdown_connections 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.135+0000 7ff22da60640 1 -- 172.21.15.62:0/3164861502 wait complete. 2024-08-16T19:16:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.135+0000 7ff22da60640 1 Processor -- start 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.135+0000 7ff22da60640 1 -- start start 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff228102260 0x7ff228074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2281078c0 0x7ff228074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111cc0 con 0x7ff2281018c0 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111b40 con 0x7ff228102260 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff22da60640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff228111e40 con 0x7ff2281078c0 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33784/0 (socket says 172.21.15.62:33784) 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 -- 172.21.15.62:0/3933506386 learned_addr learned my addr 172.21.15.62:0/3933506386 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff2277fe640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2281078c0 0x7ff228074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 -- 172.21.15.62:0/3933506386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2281078c0 msgr2=0x7ff228074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2281078c0 0x7ff228074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 -- 172.21.15.62:0/3933506386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff228102260 msgr2=0x7ff228074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff228102260 0x7ff228074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 -- 172.21.15.62:0/3933506386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2281a65e0 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.136+0000 7ff226ffd640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff218005e30 tx=0x7ff218032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.137+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218004040 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.137+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2281a6870 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.137+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2180046c0 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.137+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218046480 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.137+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2281a6e00 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.138+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff228101d20 con 0x7ff2281018c0 2024-08-16T19:16:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.139+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff2180305a0 con 0x7ff2281018c0 2024-08-16T19:16:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.140+0000 7ff22ca5e640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 0x7ff2140a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.140+0000 7ff2267fc640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 0x7ff2140a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.140+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7ff21802e030 con 0x7ff2281018c0 2024-08-16T19:16:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.141+0000 7ff2267fc640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 0x7ff2140a5230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2100058c0 tx=0x7ff210005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.144+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff21804b060 con 0x7ff2281018c0 2024-08-16T19:16:30.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.373+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff228075750 con 0x7ff2281018c0 2024-08-16T19:16:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.374+0000 7ff22ca5e640 1 -- 172.21.15.62:0/3933506386 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12000 (secure 0 0 0) 0x7ff2180bd720 con 0x7ff2281018c0 2024-08-16T19:16:30.375 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:16:30.375 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"btime":"2024-08-16T19:16:26:496652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:16:25.577802+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:16:30.375 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 39 2024-08-16T19:16:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 msgr2=0x7ff2140a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:30.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 0x7ff2140a5230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2100058c0 tx=0x7ff210005850 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 msgr2=0x7ff228074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff218005e30 tx=0x7ff218032fb0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 shutdown_connections 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff2140a2d70 0x7ff2140a5230 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2281078c0 0x7ff228074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff228102260 0x7ff228074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 --2- 172.21.15.62:0/3933506386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2281018c0 0x7ff228074220 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 >> 172.21.15.62:0/3933506386 conn(0x7ff2280fd5e0 msgr2=0x7ff2280fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 shutdown_connections 2024-08-16T19:16:30.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:30.377+0000 7ff22da60640 1 -- 172.21.15.62:0/3933506386 wait complete. 2024-08-16T19:16:30.678 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:30 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3933506386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:30.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:30 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3933506386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:30.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:30 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3933506386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:30.962 INFO:tasks.workunit.client.0.smithi062.stdout: 1 318107 5.47 MB/sec execute 169 sec latency 16.601 ms 2024-08-16T19:16:31.117 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-08-16T19:16:31.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-08-16T19:16:31.391 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:31.768 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:31 smithi062 ceph-mon[32993]: pgmap v184: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-08-16T19:16:31.768 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:31 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:31.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:31 smithi155 ceph-mon[37413]: pgmap v184: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-08-16T19:16:31.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:31 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:31.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 -- 172.21.15.62:0/2581997466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c102ac0 msgr2=0x7fc56c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2581997466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c102ac0 0x7fc56c102ea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc568009930 tx=0x7fc56802a440 comp rx=0 tx=0).stop 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 -- 172.21.15.62:0/2581997466 shutdown_connections 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2581997466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c107fc0 0x7fc56c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2581997466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc56c103470 0x7fc56c1078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2581997466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c102ac0 0x7fc56c102ea0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 -- 172.21.15.62:0/2581997466 >> 172.21.15.62:0/2581997466 conn(0x7fc56c0fc500 msgr2=0x7fc56c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 -- 172.21.15.62:0/2581997466 shutdown_connections 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 -- 172.21.15.62:0/2581997466 wait complete. 2024-08-16T19:16:31.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.864+0000 7fc5738a3640 1 Processor -- start 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 -- start start 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c102ac0 0x7fc56c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc56c103470 0x7fc56c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c111e20 con 0x7fc56c107fc0 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c111ca0 con 0x7fc56c103470 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc5738a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c111fa0 con 0x7fc56c102ac0 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46882/0 (socket says 172.21.15.62:46882) 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 -- 172.21.15.62:0/2011920273 learned_addr learned my addr 172.21.15.62:0/2011920273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc57163f640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c102ac0 0x7fc56c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 -- 172.21.15.62:0/2011920273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c102ac0 msgr2=0x7fc56c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c102ac0 0x7fc56c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 -- 172.21.15.62:0/2011920273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc56c103470 msgr2=0x7fc56c198890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc56c103470 0x7fc56c198890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 -- 172.21.15.62:0/2011920273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c19d300 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.865+0000 7fc571e40640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b990 tx=0x7fc56400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.866+0000 7fc5627fc640 1 -- 172.21.15.62:0/2011920273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564019070 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.866+0000 7fc5627fc640 1 -- 172.21.15.62:0/2011920273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc56400cba0 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.866+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c19d5f0 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.866+0000 7fc5627fc640 1 -- 172.21.15.62:0/2011920273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564007650 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.866+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c1a4ed0 con 0x7fc56c107fc0 2024-08-16T19:16:31.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.868+0000 7fc5627fc640 1 -- 172.21.15.62:0/2011920273 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc56401c070 con 0x7fc56c107fc0 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 Processor -- start 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 -- start start 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7d1f810 con 0x55b5a7dc98b0 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7cc6c70 con 0x55b5a7dbaac0 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc575ca1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7d1b7d0 con 0x55b5a7dba1a0 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc5627fc640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 0x7fc5580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc57163f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:33638/0 (socket says 172.21.15.62:33638) 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc570e3e640 1 -- 172.21.15.62:0/543684049 learned_addr learned my addr 172.21.15.62:0/543684049 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc570e3e640 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc5627fc640 1 -- 172.21.15.62:0/2011920273 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fc5640ced10 con 0x7fc56c107fc0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.869+0000 7fc571e40640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 0x7fc5580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 -- 172.21.15.62:0/543684049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 msgr2=0x55b5a7dc9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 -- 172.21.15.62:0/543684049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 msgr2=0x55b5a7dcbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 -- 172.21.15.62:0/543684049 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a7a91ba0 con 0x55b5a7dba1a0 2024-08-16T19:16:31.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc571e40640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 0x7fc5580a51b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5640cc1e0 tx=0x7fc564014c20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc570e3e640 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0058c0 tx=0x7fc55c00d380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2e63a0c064912d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc5577fe640 1 -- 172.21.15.62:0/543684049 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00a650 con 0x55b5a7dba1a0 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc5577fe640 1 -- 172.21.15.62:0/543684049 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c00acd0 con 0x55b5a7dba1a0 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.870+0000 7fc5577fe640 1 -- 172.21.15.62:0/543684049 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c010b60 con 0x55b5a7dba1a0 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 -- 172.21.15.62:0/543684049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 msgr2=0x55b5a7dba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0058c0 tx=0x7fc55c00d380 comp rx=0 tx=0).stop 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 -- 172.21.15.62:0/543684049 shutdown_connections 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 --2- 172.21.15.62:0/543684049 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dba580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 -- 172.21.15.62:0/543684049 >> 172.21.15.62:0/543684049 conn(0x55b5a7db6850 msgr2=0x55b5a7db6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 -- 172.21.15.62:0/543684049 shutdown_connections 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.871+0000 7fc575ca1740 1 -- 172.21.15.62:0/543684049 wait complete. 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 -- start start 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 Processor -- start 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dc7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7dcc480 con 0x55b5a7dba1a0 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7cda9a0 con 0x55b5a7dbaac0 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b5a7dcc600 con 0x55b5a7dc98b0 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc575ca1740 10 client.0 _ll_register_callbacks cb 0x55b5a7cc8c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:31.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46910/0 (socket says 172.21.15.62:46910) 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc570e3e640 1 -- 172.21.15.62:0/3137241707 learned_addr learned my addr 172.21.15.62:0/3137241707 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.872+0000 7fc570e3e640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dc7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc57163f640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 -- 172.21.15.62:0/3137241707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 msgr2=0x55b5a7dc7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dc7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 -- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 msgr2=0x55b5a7dc6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 -- 172.21.15.62:0/3137241707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a7ce6220 con 0x55b5a7dba1a0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc570e3e640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00afc0 tx=0x7fc55c0124e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e9a248e3949c3b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c026030 con 0x55b5a7dba1a0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fc55c004460 con 0x55b5a7dba1a0 2024-08-16T19:16:31.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc55c00acd0 con 0x55b5a7dba1a0 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc55c032050 con 0x55b5a7dba1a0 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc575ca1740 10 client.15828 fetch_fsmap learned FSMap version 40 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.873+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5a7dd8170 con 0x55b5a7dba1a0 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c00acd0 con 0x55b5a7dba1a0 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fc55c005450 con 0x55b5a7dba1a0 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 10 client.15828 fetch_fsmap finished waiting for FSMap version 40 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 10 client.15828 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 20 client.15828 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7ddb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 4 client.15828 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:16:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a7d2cf40 con 0x55b5a7dd95d0 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.874+0000 7fc57163f640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7ddb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.875+0000 7fc57163f640 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7ddb9c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f155c964aa425f server_cookie=ab87500d5316c98a in_seq=0 out_seq=0 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.875+0000 7fc554ff9640 10 client.15828 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.875+0000 7fc554ff9640 1 -- 172.21.15.62:0/3137241707 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5a7d2cf40 con 0x55b5a7dd95d0 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.875+0000 7fc554ff9640 10 client.15828 handle_command_reply: tid=0 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 client.15828 shutdown 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 msgr2=0x55b5a7dc6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00afc0 tx=0x7fc55c0124e0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 shutdown_connections 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7ddb9c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc554ff9640 0 client.15828 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:31.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dc7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dc6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 --2- 172.21.15.62:0/3137241707 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dc6790 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 >> 172.21.15.62:0/3137241707 conn(0x55b5a7db6850 msgr2=0x55b5a7db5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 shutdown_connections 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 1 -- 172.21.15.62:0/3137241707 wait complete. 2024-08-16T19:16:31.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.876+0000 7fc575ca1740 20 client.15828 trim_cache size 0 max 16384 2024-08-16T19:16:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:31 smithi161 ceph-mon[37576]: pgmap v184: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.1 MiB/s wr, 102 op/s 2024-08-16T19:16:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:31 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:31.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.893+0000 7fc575ca1740 1 Processor -- start 2024-08-16T19:16:31.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.893+0000 7fc575ca1740 1 -- start start 2024-08-16T19:16:31.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dba1a0 0x55b5a7de1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dbbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1acbb0 con 0x55b5a7dbaac0 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1ae810 con 0x55b5a7dba1a0 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc575ca1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1ad280 con 0x55b5a7dc98b0 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46920/0 (socket says 172.21.15.62:46920) 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 -- 172.21.15.62:0/2580328483 learned_addr learned my addr 172.21.15.62:0/2580328483 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:31.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc570e3e640 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dbbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc570e3e640 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dba1a0 0x55b5a7de1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 -- 172.21.15.62:0/2580328483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 msgr2=0x55b5a7dbbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dbbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 -- 172.21.15.62:0/2580328483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dba1a0 msgr2=0x55b5a7de1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dba1a0 0x55b5a7de1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 -- 172.21.15.62:0/2580328483 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a7dba8f0 con 0x55b5a7dbaac0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc57163f640 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc56802a960 tx=0x7fc5680058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16187269659d6750 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc556ffd640 1 -- 172.21.15.62:0/2580328483 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56804a070 con 0x55b5a7dbaac0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc556ffd640 1 -- 172.21.15.62:0/2580328483 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc568005500 con 0x55b5a7dbaac0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.894+0000 7fc556ffd640 1 -- 172.21.15.62:0/2580328483 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568035040 con 0x55b5a7dbaac0 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 -- 172.21.15.62:0/2580328483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 msgr2=0x55b5a7de1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc56802a960 tx=0x7fc5680058d0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 -- 172.21.15.62:0/2580328483 shutdown_connections 2024-08-16T19:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dbbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dbaac0 0x55b5a7de1560 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 --2- 172.21.15.62:0/2580328483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dba1a0 0x55b5a7de1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 -- 172.21.15.62:0/2580328483 >> 172.21.15.62:0/2580328483 conn(0x55b5a7db6850 msgr2=0x55b5a7db1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 -- 172.21.15.62:0/2580328483 shutdown_connections 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.895+0000 7fc575ca1740 1 -- 172.21.15.62:0/2580328483 wait complete. 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 -- start start 2024-08-16T19:16:31.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 Processor -- start 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dd2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dcd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1ae080 con 0x55b5a7dc98b0 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1ad980 con 0x55b5a7dbaac0 2024-08-16T19:16:31.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c1aab30 con 0x55b5a7dba1a0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc575ca1740 10 client.0 _ll_register_callbacks cb 0x7fc55c00dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc570e3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46930/0 (socket says 172.21.15.62:46930) 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc570e3e640 1 -- 172.21.15.62:0/976198343 learned_addr learned my addr 172.21.15.62:0/976198343 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc570e3e640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dd2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.896+0000 7fc57163f640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dcd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 -- 172.21.15.62:0/976198343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 msgr2=0x55b5a7dd2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dd2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 -- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 msgr2=0x55b5a7dcd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dcd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 -- 172.21.15.62:0/976198343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a7e42ff0 con 0x55b5a7dc98b0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc570e3e640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc55c010aa0 tx=0x7fc55c010580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9812623560348e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:31.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc554ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c026070 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fc55c00d9a0 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc568005500 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc55c00d4c0 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc575ca1740 10 client.15834 fetch_fsmap learned FSMap version 40 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5a7e44150 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.897+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc568005500 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7fc55c012150 con 0x55b5a7dc98b0 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 10 client.15834 fetch_fsmap finished waiting for FSMap version 40 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 10 client.15834 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 20 client.15834 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7e46980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:31.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 4 client.15834 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-08-16T19:16:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x55b5a7d27290 con 0x55b5a7dd95d0 2024-08-16T19:16:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc57163f640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7e46980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc554ff9640 10 client.15834 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:31.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.898+0000 7fc57163f640 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7e46980 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fda2ebfdafe25d3 server_cookie=a9af2d9e30fdc4ea in_seq=0 out_seq=0 2024-08-16T19:16:31.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.51 MB/sec execute 170 sec latency 60.463 ms 2024-08-16T19:16:31.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc554ff9640 1 -- 172.21.15.62:0/976198343 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b5a7d27290 con 0x55b5a7dd95d0 2024-08-16T19:16:31.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc554ff9640 10 client.15834 handle_command_reply: tid=0 2024-08-16T19:16:31.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 client.15834 shutdown 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 msgr2=0x55b5a7dcdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc55c010aa0 tx=0x7fc55c010580 comp rx=0 tx=0).stop 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 shutdown_connections 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b5a7dd95d0 0x55b5a7e46980 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b5a7dc98b0 0x55b5a7dcdcb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5a7dbaac0 0x55b5a7dcd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 --2- 172.21.15.62:0/976198343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b5a7dba1a0 0x55b5a7dd2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 >> 172.21.15.62:0/976198343 conn(0x55b5a7db6850 msgr2=0x55b5a7dbbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc554ff9640 0 client.15834 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 shutdown_connections 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 1 -- 172.21.15.62:0/976198343 wait complete. 2024-08-16T19:16:31.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.974+0000 7fc575ca1740 20 client.15834 trim_cache size 0 max 16384 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 40, 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 26, 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "3c8204b": { 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 26, 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:31.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 141.1, 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:31.978 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 msgr2=0x7fc5580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 0x7fc5580a51b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5640cc1e0 tx=0x7fc564014c20 comp rx=0 tx=0).stop 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 msgr2=0x7fc56c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc56400b990 tx=0x7fc56400be50 comp rx=0 tx=0).stop 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 shutdown_connections 2024-08-16T19:16:31.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc5580a2cf0 0x7fc5580a51b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc56c107fc0 0x7fc56c19cc20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc56c103470 0x7fc56c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 --2- 172.21.15.62:0/2011920273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc56c102ac0 0x7fc56c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 >> 172.21.15.62:0/2011920273 conn(0x7fc56c0fc500 msgr2=0x7fc56c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 shutdown_connections 2024-08-16T19:16:31.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:31.975+0000 7fc5738a3640 1 -- 172.21.15.62:0/2011920273 wait complete. 2024-08-16T19:16:32.929 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3c8204b', quiesce took 3.5 seconds. Will release after: 17.55 2024-08-16T19:16:32.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.48 MB/sec execute 171 sec latency 1050.007 ms 2024-08-16T19:16:33.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:33 smithi155 ceph-mon[37413]: pgmap v185: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-08-16T19:16:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:33 smithi161 ceph-mon[37576]: pgmap v185: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-08-16T19:16:33.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.45 MB/sec execute 172 sec latency 2050.124 ms 2024-08-16T19:16:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:33 smithi062 ceph-mon[32993]: pgmap v185: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-08-16T19:16:34.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.42 MB/sec execute 173 sec latency 3050.225 ms 2024-08-16T19:16:35.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:35 smithi062 ceph-mon[32993]: pgmap v186: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-08-16T19:16:35.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:35 smithi155 ceph-mon[37413]: pgmap v186: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-08-16T19:16:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:35 smithi161 ceph-mon[37576]: pgmap v186: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.3 MiB/s wr, 156 op/s 2024-08-16T19:16:35.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.38 MB/sec execute 174 sec latency 4050.346 ms 2024-08-16T19:16:36.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:36 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:16:36.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:36 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:16:36.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.35 MB/sec execute 175 sec latency 5050.448 ms 2024-08-16T19:16:36.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:36 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:16:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:37 smithi155 ceph-mon[37413]: pgmap v187: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:16:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:16:37.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:16:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:37 smithi161 ceph-mon[37576]: pgmap v187: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:16:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:16:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:16:37.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.32 MB/sec execute 176 sec latency 6050.549 ms 2024-08-16T19:16:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:37 smithi062 ceph-mon[32993]: pgmap v187: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:16:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:16:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:16:38.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.29 MB/sec execute 177 sec latency 7050.653 ms 2024-08-16T19:16:39.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:39 smithi155 ceph-mon[37413]: pgmap v188: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:39 smithi161 ceph-mon[37576]: pgmap v188: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:39.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.26 MB/sec execute 178 sec latency 8050.786 ms 2024-08-16T19:16:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:39 smithi062 ceph-mon[32993]: pgmap v188: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-08-16T19:16:40.963 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.23 MB/sec execute 179 sec latency 9050.854 ms 2024-08-16T19:16:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:41 smithi155 ceph-mon[37413]: pgmap v189: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-16T19:16:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:41 smithi161 ceph-mon[37576]: pgmap v189: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-16T19:16:41.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.20 MB/sec execute 180 sec latency 10050.960 ms 2024-08-16T19:16:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:41 smithi062 ceph-mon[32993]: pgmap v189: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-16T19:16:42.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:42 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:42.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:42 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:42.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.18 MB/sec execute 181 sec latency 11051.067 ms 2024-08-16T19:16:42.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:42 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:43.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:43 smithi155 ceph-mon[37413]: pgmap v190: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-08-16T19:16:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:43 smithi161 ceph-mon[37576]: pgmap v190: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-08-16T19:16:43.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.15 MB/sec execute 182 sec latency 12051.160 ms 2024-08-16T19:16:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:43 smithi062 ceph-mon[32993]: pgmap v190: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1018 KiB/s wr, 104 op/s 2024-08-16T19:16:44.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.12 MB/sec execute 183 sec latency 13051.265 ms 2024-08-16T19:16:45.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:45 smithi155 ceph-mon[37413]: pgmap v191: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-08-16T19:16:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:45 smithi161 ceph-mon[37576]: pgmap v191: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-08-16T19:16:45.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.09 MB/sec execute 184 sec latency 14051.375 ms 2024-08-16T19:16:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:45 smithi062 ceph-mon[32993]: pgmap v191: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-08-16T19:16:46.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.06 MB/sec execute 185 sec latency 15051.516 ms 2024-08-16T19:16:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:47 smithi155 ceph-mon[37413]: pgmap v192: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:47 smithi161 ceph-mon[37576]: pgmap v192: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:47.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.04 MB/sec execute 186 sec latency 16051.630 ms 2024-08-16T19:16:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:47 smithi062 ceph-mon[32993]: pgmap v192: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:48.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 5.01 MB/sec execute 187 sec latency 17051.743 ms 2024-08-16T19:16:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:49 smithi155 ceph-mon[37413]: pgmap v193: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:49 smithi161 ceph-mon[37576]: pgmap v193: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:49.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 4.98 MB/sec execute 188 sec latency 18051.810 ms 2024-08-16T19:16:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:49 smithi062 ceph-mon[32993]: pgmap v193: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 338 KiB/s wr, 39 op/s 2024-08-16T19:16:50.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:16:50.760 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:50.964 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 4.96 MB/sec execute 189 sec latency 19051.924 ms 2024-08-16T19:16:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 -- 172.21.15.62:0/3724071758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 msgr2=0x7fd81810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 --2- 172.21.15.62:0/3724071758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd81810d770 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd810009ca0 tx=0x7fd81002a950 comp rx=0 tx=0).stop 2024-08-16T19:16:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 -- 172.21.15.62:0/3724071758 shutdown_connections 2024-08-16T19:16:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 --2- 172.21.15.62:0/3724071758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd81810d770 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 --2- 172.21.15.62:0/3724071758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 0x7fd8181026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 --2- 172.21.15.62:0/3724071758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 0x7fd818100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 -- 172.21.15.62:0/3724071758 >> 172.21.15.62:0/3724071758 conn(0x7fd8180fc500 msgr2=0x7fd8180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 -- 172.21.15.62:0/3724071758 shutdown_connections 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.238+0000 7fd81c831640 1 -- 172.21.15.62:0/3724071758 wait complete. 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 Processor -- start 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 -- start start 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 0x7fd818199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 0x7fd8181995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110cf0 con 0x7fd818102c20 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110b70 con 0x7fd8181022a0 2024-08-16T19:16:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81c831640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110e70 con 0x7fd818100b50 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55148/0 (socket says 172.21.15.62:55148) 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 -- 172.21.15.62:0/3347051956 learned_addr learned my addr 172.21.15.62:0/3347051956 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd815d9b640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 0x7fd8181995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd81659c640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 0x7fd818199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 -- 172.21.15.62:0/3347051956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 msgr2=0x7fd818199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 0x7fd818199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 -- 172.21.15.62:0/3347051956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 msgr2=0x7fd8181995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 0x7fd8181995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.239+0000 7fd816d9d640 1 -- 172.21.15.62:0/3347051956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8181a4620 con 0x7fd818102c20 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd816d9d640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd810006010 tx=0x7fd81002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810034040 con 0x7fd818102c20 2024-08-16T19:16:51.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd818196e50 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd810003e00 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810045480 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.240+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd818197320 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.241+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd818100f30 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.242+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd810037020 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.243+0000 7fd8077fe640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 0x7fd8000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.243+0000 7fd81659c640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 0x7fd8000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.244+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7fd8100f0770 con 0x7fd818102c20 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.244+0000 7fd81659c640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 0x7fd8000a51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8080058c0 tx=0x7fd80800a250 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.248+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd8100bd110 con 0x7fd818102c20 2024-08-16T19:16:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.463+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd818197580 con 0x7fd818102c20 2024-08-16T19:16:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.464+0000 7fd8077fe640 1 -- 172.21.15.62:0/3347051956 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12000 (secure 0 0 0) 0x7fd8100bca10 con 0x7fd818102c20 2024-08-16T19:16:51.466 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:16:51.466 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":41,"btime":"2024-08-16T19:16:41:528074+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:16:40.578298+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:16:51.466 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 41 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 msgr2=0x7fd8000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 0x7fd8000a51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8080058c0 tx=0x7fd80800a250 comp rx=0 tx=0).stop 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 msgr2=0x7fd8181a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd810006010 tx=0x7fd81002f6f0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 shutdown_connections 2024-08-16T19:16:51.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fd8000a2cf0 0x7fd8000a51b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd818102c20 0x7fd8181a40e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8181022a0 0x7fd8181995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 --2- 172.21.15.62:0/3347051956 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd818100b50 0x7fd818199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.466+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 >> 172.21.15.62:0/3347051956 conn(0x7fd8180fc500 msgr2=0x7fd818106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.467+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 shutdown_connections 2024-08-16T19:16:51.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:51.467+0000 7fd81c831640 1 -- 172.21.15.62:0/3347051956 wait complete. 2024-08-16T19:16:51.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:51 smithi062 ceph-mon[32993]: pgmap v194: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 353 KiB/s wr, 45 op/s 2024-08-16T19:16:51.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:51 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3347051956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:51 smithi155 ceph-mon[37413]: pgmap v194: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 353 KiB/s wr, 45 op/s 2024-08-16T19:16:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:51 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3347051956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:51 smithi161 ceph-mon[37576]: pgmap v194: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 353 KiB/s wr, 45 op/s 2024-08-16T19:16:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:51 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3347051956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:16:51.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319669 4.93 MB/sec execute 190 sec latency 20051.970 ms 2024-08-16T19:16:52.063 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 3c8204b --release --await' 2024-08-16T19:16:52.064 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 3c8204b --release --await 2024-08-16T19:16:52.332 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:16:52.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2209770432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 msgr2=0x7f8c1c074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2209770432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c074730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009900 tx=0x7f8c0c02a480 comp rx=0 tx=0).stop 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2209770432 shutdown_connections 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2209770432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 0x7f8c1c110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2209770432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 0x7f8c1c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2209770432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c074730 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2209770432 >> 172.21.15.62:0/2209770432 conn(0x7f8c1c0fd5f0 msgr2=0x7f8c1c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2209770432 shutdown_connections 2024-08-16T19:16:52.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2209770432 wait complete. 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.788+0000 7f8c20ca4640 1 Processor -- start 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 -- start start 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 0x7f8c1c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 0x7f8c1c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c113ea0 con 0x7f8c1c074350 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c113d20 con 0x7f8c1c074c70 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c20ca4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c114020 con 0x7f8c1c10e0f0 2024-08-16T19:16:52.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55160/0 (socket says 172.21.15.62:55160) 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 -- 172.21.15.62:0/2964015300 learned_addr learned my addr 172.21.15.62:0/2964015300 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 0x7f8c1c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 0x7f8c1c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 -- 172.21.15.62:0/2964015300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 msgr2=0x7f8c1c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 0x7f8c1c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 -- 172.21.15.62:0/2964015300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 msgr2=0x7f8c1c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 0x7f8c1c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.789+0000 7f8c1a59c640 1 -- 172.21.15.62:0/2964015300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c19f2e0 con 0x7f8c1c074350 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006010 tx=0x7f8c0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c0b7fe640 1 -- 172.21.15.62:0/2964015300 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c03f030 con 0x7f8c1c074350 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c19f570 con 0x7f8c1c074350 2024-08-16T19:16:52.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c0b7fe640 1 -- 172.21.15.62:0/2964015300 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c003ce0 con 0x7f8c1c074350 2024-08-16T19:16:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c0b7fe640 1 -- 172.21.15.62:0/2964015300 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c049060 con 0x7f8c1c074350 2024-08-16T19:16:52.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.790+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c19f9e0 con 0x7f8c1c074350 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.792+0000 7f8c0b7fe640 1 -- 172.21.15.62:0/2964015300 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8c0c042030 con 0x7f8c1c074350 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 Processor -- start 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 -- start start 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1d9d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5617c1cf8c60 con 0x5617c1d92990 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5617c1c90b90 con 0x5617c1d932b0 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c230a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617c1cf4c20 con 0x5617c1d9db70 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c0b7fe640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 0x7f8c040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c1ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c1ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55166/0 (socket says 172.21.15.62:55166) 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c1ad9d640 1 -- 172.21.15.62:0/1317275928 learned_addr learned my addr 172.21.15.62:0/1317275928 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1d9d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c0b7fe640 1 -- 172.21.15.62:0/2964015300 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f8c0c0ef5c0 con 0x7f8c1c074350 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.793+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 0x7f8c040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 -- 172.21.15.62:0/1317275928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 msgr2=0x5617c1d9ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 -- 172.21.15.62:0/1317275928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 msgr2=0x5617c1d9d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1d9d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 -- 172.21.15.62:0/1317275928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c1a08580 con 0x5617c1d92990 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8c14014a60 tx=0x7f8c14014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dd7135f8f411e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c08ff9640 1 -- 172.21.15.62:0/1317275928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1401ec50 con 0x5617c1d92990 2024-08-16T19:16:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c08ff9640 1 -- 172.21.15.62:0/1317275928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14023070 con 0x5617c1d92990 2024-08-16T19:16:52.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.794+0000 7f8c08ff9640 1 -- 172.21.15.62:0/1317275928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140056b0 con 0x5617c1d92990 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- 172.21.15.62:0/1317275928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 msgr2=0x5617c1d92d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 0x7f8c040a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c14022000 tx=0x7f8c1401e000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8c14014a60 tx=0x7f8c14014f20 comp rx=0 tx=0).stop 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- 172.21.15.62:0/1317275928 shutdown_connections 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1d9d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1317275928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1d92d70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- 172.21.15.62:0/1317275928 >> 172.21.15.62:0/1317275928 conn(0x5617c1d8ef20 msgr2=0x5617c1d8f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- 172.21.15.62:0/1317275928 shutdown_connections 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- 172.21.15.62:0/1317275928 wait complete. 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.795+0000 7f8c230a2740 1 -- start start 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 Processor -- start 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d92990 0x5617c1d9aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5617c1da0740 con 0x5617c1d932b0 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5617c1c967c0 con 0x5617c1d92990 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617c1da08c0 con 0x5617c1d9db70 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55174/0 (socket says 172.21.15.62:55174) 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 -- 172.21.15.62:0/3044545307 learned_addr learned my addr 172.21.15.62:0/3044545307 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c230a2740 10 client.0 _ll_register_callbacks cb 0x5617c1c9ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d92990 0x5617c1d9aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 -- 172.21.15.62:0/3044545307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 msgr2=0x5617c1d9b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 -- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d92990 msgr2=0x5617c1d9aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d92990 0x5617c1d9aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 -- 172.21.15.62:0/3044545307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c1ad2960 con 0x5617c1d932b0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8c10010850 tx=0x7f8c10010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be9d435d930b08c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.796+0000 7f8c01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000fb90 con 0x5617c1d932b0 2024-08-16T19:16:52.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f8c1000fd30 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c14023070 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c10024050 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 10 client.15852 fetch_fsmap learned FSMap version 41 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5617c1dac7c0 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c14023070 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f8c10018510 con 0x5617c1d932b0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 41 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 10 client.15852 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 20 client.15852 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1dadcf0 0x5617c1db01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 4 client.15852 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.797+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617c1d061d0 con 0x5617c1dadcf0 2024-08-16T19:16:52.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.798+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1dadcf0 0x5617c1db01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.798+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1dadcf0 0x5617c1db01b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1462087b8eadfcc2 server_cookie=272b662187cf55a9 in_seq=0 out_seq=0 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.798+0000 7f8c01ffb640 10 client.15852 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c01ffb640 1 -- 172.21.15.62:0/3044545307 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617c1d061d0 con 0x5617c1dadcf0 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c01ffb640 10 client.15852 handle_command_reply: tid=0 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 client.15852 shutdown 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 msgr2=0x5617c1d9af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8c10010850 tx=0x7f8c10010d10 comp rx=0 tx=0).stop 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 shutdown_connections 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1dadcf0 0x5617c1db01b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d9b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1d9af90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c01ffb640 0 client.15852 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 --2- 172.21.15.62:0/3044545307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d92990 0x5617c1d9aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 >> 172.21.15.62:0/3044545307 conn(0x5617c1d8ef20 msgr2=0x5617c1d8e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 shutdown_connections 2024-08-16T19:16:52.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 1 -- 172.21.15.62:0/3044545307 wait complete. 2024-08-16T19:16:52.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.799+0000 7f8c230a2740 20 client.15852 trim_cache size 0 max 16384 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 Processor -- start 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 -- start start 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1db6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d97cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b0f90 con 0x5617c1d92990 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b2bc0 con 0x5617c1d932b0 2024-08-16T19:16:52.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c230a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b1660 con 0x5617c1d9db70 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55178/0 (socket says 172.21.15.62:55178) 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 -- 172.21.15.62:0/848397925 learned_addr learned my addr 172.21.15.62:0/848397925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d97cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1db6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 -- 172.21.15.62:0/848397925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 msgr2=0x5617c1d97cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d97cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 -- 172.21.15.62:0/848397925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 msgr2=0x5617c1db6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1db6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 -- 172.21.15.62:0/848397925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c1d8e9c0 con 0x5617c1d92990 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d97cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1db6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:16:52.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.817+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0eca30 tx=0x7f8c0c08d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2ec22910dfce8c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c03fff640 1 -- 172.21.15.62:0/848397925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c035040 con 0x5617c1d92990 2024-08-16T19:16:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c03fff640 1 -- 172.21.15.62:0/848397925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c03b460 con 0x5617c1d92990 2024-08-16T19:16:52.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c03fff640 1 -- 172.21.15.62:0/848397925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0ecc00 con 0x5617c1d92990 2024-08-16T19:16:52.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 -- 172.21.15.62:0/848397925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 msgr2=0x5617c1db5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0eca30 tx=0x7f8c0c08d710 comp rx=0 tx=0).stop 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 -- 172.21.15.62:0/848397925 shutdown_connections 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d9db70 0x5617c1d97cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d932b0 0x5617c1db6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 --2- 172.21.15.62:0/848397925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d92990 0x5617c1db5b10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 -- 172.21.15.62:0/848397925 >> 172.21.15.62:0/848397925 conn(0x5617c1d8ef20 msgr2=0x5617c1d99320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 -- 172.21.15.62:0/848397925 shutdown_connections 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.818+0000 7f8c230a2740 1 -- 172.21.15.62:0/848397925 wait complete. 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 -- start start 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 Processor -- start 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d92990 0x5617c1da1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d9db70 0x5617c1da2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b2430 con 0x5617c1d932b0 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1b1d60 con 0x5617c1d9db70 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1aef10 con 0x5617c1d92990 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c230a2740 10 client.0 _ll_register_callbacks cb 0x7f8c10010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55192/0 (socket says 172.21.15.62:55192) 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d92990 0x5617c1da1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 -- 172.21.15.62:0/1723339858 learned_addr learned my addr 172.21.15.62:0/1723339858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c1ad9d640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d9db70 0x5617c1da2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 -- 172.21.15.62:0/1723339858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d92990 msgr2=0x5617c1da1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d92990 0x5617c1da1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 -- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d9db70 msgr2=0x5617c1da2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d9db70 0x5617c1da2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.819+0000 7f8c19d9b640 1 -- 172.21.15.62:0/1723339858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c1e17ae0 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c19d9b640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5617c1d9c090 tx=0x7f8c10018e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c59d6f64af9bc7e server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000a340 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9437+0+0 (secure 0 0 0) 0x7f8c1000fda0 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617c1d006e0 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c10036050 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 10 client.15858 fetch_fsmap learned FSMap version 41 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5617c1e18dd0 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5617c1d006e0 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f8c10004660 con 0x5617c1d932b0 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 10 client.15858 fetch_fsmap finished waiting for FSMap version 41 2024-08-16T19:16:52.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 10 client.15858 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 20 client.15858 populate_metadata read hostname 'smithi062' 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1e19630 0x5617c1e1ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 4 client.15858 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3c8204b", "release": true, "await": true}] 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.820+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3c8204b", "release": true, "await": true}) -- 0x5617c1c7d5a0 con 0x5617c1e19630 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.821+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1e19630 0x5617c1e1ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.821+0000 7f8c1a59c640 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1e19630 0x5617c1e1ba50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d1f775ee534d6d server_cookie=2cab02c7ebdbd6ee in_seq=0 out_seq=0 2024-08-16T19:16:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.821+0000 7f8c01ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c01ffb640 1 -- 172.21.15.62:0/1723339858 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5617c1c7d5a0 con 0x5617c1e19630 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c01ffb640 10 client.15858 handle_command_reply: tid=0 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 client.15858 shutdown 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 msgr2=0x5617c1da1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5617c1d9c090 tx=0x7f8c10018e50 comp rx=0 tx=0).stop 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 shutdown_connections 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5617c1e19630 0x5617c1e1ba50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617c1d9db70 0x5617c1da2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5617c1d932b0 0x5617c1da1dd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c01ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 --2- 172.21.15.62:0/1723339858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617c1d92990 0x5617c1da1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 >> 172.21.15.62:0/1723339858 conn(0x5617c1d8ef20 msgr2=0x5617c1d8d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 shutdown_connections 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 1 -- 172.21.15.62:0/1723339858 wait complete. 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.829+0000 7f8c230a2740 20 client.15858 trim_cache size 0 max 16384 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 41, 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 28, 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:16:52.832 INFO:teuthology.orchestra.run.smithi062.stdout: "3c8204b": { 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 28, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 141.1, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:16:52.833 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:16:52.834 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 msgr2=0x7f8c040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 0x7f8c040a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c14022000 tx=0x7f8c1401e000 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 msgr2=0x7f8c1c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006010 tx=0x7f8c0c0042f0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 shutdown_connections 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f8c040a2cf0 0x7f8c040a51b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c10e0f0 0x7f8c1c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c1c074c70 0x7f8c1c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 --2- 172.21.15.62:0/2964015300 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c1c074350 0x7f8c1c19a330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 >> 172.21.15.62:0/2964015300 conn(0x7f8c1c0fd5f0 msgr2=0x7f8c1c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 shutdown_connections 2024-08-16T19:16:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:16:52.830+0000 7f8c20ca4640 1 -- 172.21.15.62:0/2964015300 wait complete. 2024-08-16T19:16:52.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 319820 4.91 MB/sec execute 191 sec latency 20919.155 ms 2024-08-16T19:16:53.484 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3c8204b', total seconds elapsed: 24.0 2024-08-16T19:16:53.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:53 smithi062 ceph-mon[32993]: pgmap v195: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 303 KiB/s wr, 27 op/s 2024-08-16T19:16:53.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:53 smithi155 ceph-mon[37413]: pgmap v195: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 303 KiB/s wr, 27 op/s 2024-08-16T19:16:53.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:53 smithi161 ceph-mon[37576]: pgmap v195: 129 pgs: 129 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 303 KiB/s wr, 27 op/s 2024-08-16T19:16:53.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 321842 4.92 MB/sec execute 192 sec latency 15.300 ms 2024-08-16T19:16:54.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323480 4.95 MB/sec execute 193 sec latency 9.702 ms 2024-08-16T19:16:55.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:55 smithi155 ceph-mon[37413]: pgmap v196: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 544 KiB/s wr, 65 op/s 2024-08-16T19:16:55.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:55 smithi161 ceph-mon[37576]: pgmap v196: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 544 KiB/s wr, 65 op/s 2024-08-16T19:16:55.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323584 4.93 MB/sec execute 194 sec latency 594.538 ms 2024-08-16T19:16:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:55 smithi062 ceph-mon[32993]: pgmap v196: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 544 KiB/s wr, 65 op/s 2024-08-16T19:16:56.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:56 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:56.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:56 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:56.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323584 4.90 MB/sec execute 195 sec latency 1343.028 ms 2024-08-16T19:16:56.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:56 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:16:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:57 smithi155 ceph-mon[37413]: pgmap v197: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:16:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:57 smithi161 ceph-mon[37576]: pgmap v197: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:16:57.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323584 4.88 MB/sec execute 196 sec latency 2343.156 ms 2024-08-16T19:16:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:57 smithi062 ceph-mon[32993]: pgmap v197: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:16:58.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323584 4.85 MB/sec execute 197 sec latency 3343.253 ms 2024-08-16T19:16:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:16:59 smithi155 ceph-mon[37413]: pgmap v198: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:16:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:16:59 smithi161 ceph-mon[37576]: pgmap v198: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:16:59.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 323585 4.83 MB/sec execute 198 sec latency 4336.593 ms 2024-08-16T19:16:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:16:59 smithi062 ceph-mon[32993]: pgmap v198: 129 pgs: 129 active+clean; 207 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 256 KiB/s wr, 44 op/s 2024-08-16T19:17:00.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 325243 4.84 MB/sec execute 199 sec latency 215.514 ms 2024-08-16T19:17:01.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:01 smithi155 ceph-mon[37413]: pgmap v199: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 369 KiB/s wr, 66 op/s 2024-08-16T19:17:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:01 smithi161 ceph-mon[37576]: pgmap v199: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 369 KiB/s wr, 66 op/s 2024-08-16T19:17:01.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 326895 4.88 MB/sec execute 200 sec latency 54.206 ms 2024-08-16T19:17:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:01 smithi062 ceph-mon[32993]: pgmap v199: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 369 KiB/s wr, 66 op/s 2024-08-16T19:17:02.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 327060 4.85 MB/sec execute 201 sec latency 874.035 ms 2024-08-16T19:17:03.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:03 smithi161 ceph-mon[37576]: pgmap v200: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 354 KiB/s wr, 60 op/s 2024-08-16T19:17:03.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 327060 4.83 MB/sec execute 202 sec latency 1874.161 ms 2024-08-16T19:17:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:03 smithi062 ceph-mon[32993]: pgmap v200: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 354 KiB/s wr, 60 op/s 2024-08-16T19:17:04.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:03 smithi155 ceph-mon[37413]: pgmap v200: 129 pgs: 129 active+clean; 208 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 354 KiB/s wr, 60 op/s 2024-08-16T19:17:04.965 INFO:tasks.workunit.client.0.smithi062.stdout: 1 327060 4.80 MB/sec execute 203 sec latency 2874.281 ms 2024-08-16T19:17:05.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:05 smithi161 ceph-mon[37576]: pgmap v201: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-08-16T19:17:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:05 smithi161 ceph-mon[37576]: osdmap e94: 12 total, 12 up, 12 in 2024-08-16T19:17:05.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 327772 4.80 MB/sec execute 204 sec latency 3497.972 ms 2024-08-16T19:17:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:05 smithi062 ceph-mon[32993]: pgmap v201: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-08-16T19:17:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:05 smithi062 ceph-mon[32993]: osdmap e94: 12 total, 12 up, 12 in 2024-08-16T19:17:06.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:05 smithi155 ceph-mon[37413]: pgmap v201: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-08-16T19:17:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:05 smithi155 ceph-mon[37413]: osdmap e94: 12 total, 12 up, 12 in 2024-08-16T19:17:06.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 330127 4.85 MB/sec execute 205 sec latency 12.201 ms 2024-08-16T19:17:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:07 smithi161 ceph-mon[37576]: pgmap v203: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:07.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 330552 4.83 MB/sec execute 206 sec latency 548.084 ms 2024-08-16T19:17:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:07 smithi062 ceph-mon[32993]: pgmap v203: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:08.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:07 smithi155 ceph-mon[37413]: pgmap v203: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:08.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 330552 4.80 MB/sec execute 207 sec latency 1548.190 ms 2024-08-16T19:17:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:09 smithi161 ceph-mon[37576]: pgmap v204: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:09.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 330552 4.78 MB/sec execute 208 sec latency 2548.317 ms 2024-08-16T19:17:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:09 smithi062 ceph-mon[32993]: pgmap v204: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:10.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:09 smithi155 ceph-mon[37413]: pgmap v204: 129 pgs: 129 active+clean; 210 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-16T19:17:10.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 330954 4.77 MB/sec execute 209 sec latency 3174.496 ms 2024-08-16T19:17:11.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:11 smithi161 ceph-mon[37576]: pgmap v205: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-08-16T19:17:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:11 smithi161 ceph-mon[37576]: osdmap e95: 12 total, 12 up, 12 in 2024-08-16T19:17:11.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 333494 4.82 MB/sec execute 210 sec latency 12.795 ms 2024-08-16T19:17:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:11 smithi062 ceph-mon[32993]: pgmap v205: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-08-16T19:17:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:11 smithi062 ceph-mon[32993]: osdmap e95: 12 total, 12 up, 12 in 2024-08-16T19:17:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:11 smithi155 ceph-mon[37413]: pgmap v205: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-08-16T19:17:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:11 smithi155 ceph-mon[37413]: osdmap e95: 12 total, 12 up, 12 in 2024-08-16T19:17:12.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 334701 4.81 MB/sec execute 211 sec latency 10.239 ms 2024-08-16T19:17:13.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:13 smithi155 ceph-mon[37413]: pgmap v207: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 471 KiB/s wr, 66 op/s 2024-08-16T19:17:13.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:13 smithi161 ceph-mon[37576]: pgmap v207: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 471 KiB/s wr, 66 op/s 2024-08-16T19:17:13.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 336940 4.86 MB/sec execute 212 sec latency 15.469 ms 2024-08-16T19:17:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:13 smithi062 ceph-mon[32993]: pgmap v207: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 471 KiB/s wr, 66 op/s 2024-08-16T19:17:14.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 338209 4.86 MB/sec execute 213 sec latency 12.710 ms 2024-08-16T19:17:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:15 smithi161 ceph-mon[37576]: pgmap v208: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-16T19:17:15.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 340339 4.89 MB/sec execute 214 sec latency 14.322 ms 2024-08-16T19:17:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:15 smithi062 ceph-mon[32993]: pgmap v208: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-16T19:17:16.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:15 smithi155 ceph-mon[37413]: pgmap v208: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-16T19:17:16.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341241 4.89 MB/sec execute 215 sec latency 232.259 ms 2024-08-16T19:17:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:17 smithi161 ceph-mon[37576]: pgmap v209: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:17.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341241 4.87 MB/sec execute 216 sec latency 1232.294 ms 2024-08-16T19:17:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:17 smithi062 ceph-mon[32993]: pgmap v209: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:17 smithi155 ceph-mon[37413]: pgmap v209: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:18.966 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341241 4.85 MB/sec execute 217 sec latency 2232.346 ms 2024-08-16T19:17:19.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:19 smithi161 ceph-mon[37576]: pgmap v210: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:19.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341241 4.82 MB/sec execute 218 sec latency 3232.470 ms 2024-08-16T19:17:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:19 smithi062 ceph-mon[32993]: pgmap v210: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:19 smithi155 ceph-mon[37413]: pgmap v210: 129 pgs: 129 active+clean; 220 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-08-16T19:17:20.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341309 4.80 MB/sec execute 219 sec latency 3854.183 ms 2024-08-16T19:17:21.890 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:21 smithi161 ceph-mon[37576]: pgmap v211: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-08-16T19:17:21.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341309 4.78 MB/sec execute 220 sec latency 1314.263 ms 2024-08-16T19:17:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:21 smithi062 ceph-mon[32993]: pgmap v211: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-08-16T19:17:22.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:21 smithi155 ceph-mon[37413]: pgmap v211: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-08-16T19:17:22.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341309 4.76 MB/sec execute 221 sec latency 2314.361 ms 2024-08-16T19:17:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:23 smithi161 ceph-mon[37576]: pgmap v212: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-08-16T19:17:23.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341309 4.74 MB/sec execute 222 sec latency 3314.421 ms 2024-08-16T19:17:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:23 smithi062 ceph-mon[32993]: pgmap v212: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-08-16T19:17:24.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:23 smithi155 ceph-mon[37413]: pgmap v212: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-08-16T19:17:24.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 341309 4.72 MB/sec execute 223 sec latency 4314.473 ms 2024-08-16T19:17:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:25 smithi161 ceph-mon[37576]: pgmap v213: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1008 KiB/s wr, 146 op/s 2024-08-16T19:17:25.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 342077 4.72 MB/sec execute 224 sec latency 4933.620 ms 2024-08-16T19:17:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:25 smithi062 ceph-mon[32993]: pgmap v213: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1008 KiB/s wr, 146 op/s 2024-08-16T19:17:26.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:25 smithi155 ceph-mon[37413]: pgmap v213: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1008 KiB/s wr, 146 op/s 2024-08-16T19:17:26.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 344293 4.76 MB/sec execute 225 sec latency 76.799 ms 2024-08-16T19:17:27.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:27 smithi161 ceph-mon[37576]: pgmap v214: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:27.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 345562 4.76 MB/sec execute 226 sec latency 9.930 ms 2024-08-16T19:17:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:27 smithi062 ceph-mon[32993]: pgmap v214: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:28.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:27 smithi155 ceph-mon[37413]: pgmap v214: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:28.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 347774 4.80 MB/sec execute 227 sec latency 16.149 ms 2024-08-16T19:17:29.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:29 smithi161 ceph-mon[37576]: pgmap v215: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:29.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 349013 4.80 MB/sec execute 228 sec latency 12.051 ms 2024-08-16T19:17:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:29 smithi062 ceph-mon[32993]: pgmap v215: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:30.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:29 smithi155 ceph-mon[37413]: pgmap v215: 129 pgs: 129 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 411 KiB/s wr, 65 op/s 2024-08-16T19:17:30.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 351138 4.84 MB/sec execute 229 sec latency 19.915 ms 2024-08-16T19:17:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:31 smithi161 ceph-mon[37576]: pgmap v216: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 971 KiB/s wr, 141 op/s 2024-08-16T19:17:31.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 351824 4.82 MB/sec execute 230 sec latency 365.491 ms 2024-08-16T19:17:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:31 smithi062 ceph-mon[32993]: pgmap v216: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 971 KiB/s wr, 141 op/s 2024-08-16T19:17:32.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:31 smithi155 ceph-mon[37413]: pgmap v216: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 971 KiB/s wr, 141 op/s 2024-08-16T19:17:32.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 351824 4.80 MB/sec execute 231 sec latency 1365.585 ms 2024-08-16T19:17:33.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:33 smithi161 ceph-mon[37576]: pgmap v217: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 604 KiB/s wr, 87 op/s 2024-08-16T19:17:33.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 351824 4.78 MB/sec execute 232 sec latency 2365.684 ms 2024-08-16T19:17:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:33 smithi062 ceph-mon[32993]: pgmap v217: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 604 KiB/s wr, 87 op/s 2024-08-16T19:17:34.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:33 smithi155 ceph-mon[37413]: pgmap v217: 129 pgs: 129 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 604 KiB/s wr, 87 op/s 2024-08-16T19:17:34.967 INFO:tasks.workunit.client.0.smithi062.stdout: 1 351824 4.76 MB/sec execute 233 sec latency 3365.733 ms 2024-08-16T19:17:35.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:35 smithi161 ceph-mon[37576]: pgmap v218: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-08-16T19:17:35.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 352378 4.75 MB/sec execute 234 sec latency 3985.931 ms 2024-08-16T19:17:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:35 smithi062 ceph-mon[32993]: pgmap v218: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-08-16T19:17:36.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:35 smithi155 ceph-mon[37413]: pgmap v218: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-08-16T19:17:36.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 354791 4.79 MB/sec execute 235 sec latency 21.479 ms 2024-08-16T19:17:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:37 smithi161 ceph-mon[37576]: pgmap v219: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:17:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:17:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:17:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:17:37.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 356126 4.80 MB/sec execute 236 sec latency 11.873 ms 2024-08-16T19:17:37.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:37 smithi062 ceph-mon[32993]: pgmap v219: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:37.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:17:37.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:17:37.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:17:37.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:17:38.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:37 smithi155 ceph-mon[37413]: pgmap v219: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:17:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:17:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:17:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:17:38.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358473 4.84 MB/sec execute 237 sec latency 14.169 ms 2024-08-16T19:17:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:39 smithi161 ceph-mon[37576]: pgmap v220: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:39.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358916 4.82 MB/sec execute 238 sec latency 497.656 ms 2024-08-16T19:17:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:39 smithi062 ceph-mon[32993]: pgmap v220: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:39 smithi155 ceph-mon[37413]: pgmap v220: 129 pgs: 129 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 998 KiB/s wr, 130 op/s 2024-08-16T19:17:40.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358972 4.80 MB/sec execute 239 sec latency 1119.910 ms 2024-08-16T19:17:40.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:40 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:40 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:41.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:40 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:41.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358972 4.78 MB/sec execute 240 sec latency 1326.692 ms 2024-08-16T19:17:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:41 smithi062 ceph-mon[32993]: pgmap v221: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-08-16T19:17:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:41 smithi155 ceph-mon[37413]: pgmap v221: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-08-16T19:17:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:41 smithi161 ceph-mon[37576]: pgmap v221: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-08-16T19:17:42.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358972 4.76 MB/sec execute 241 sec latency 2326.795 ms 2024-08-16T19:17:42.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:42 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:43.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:42 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:43.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:42 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:43.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358972 4.74 MB/sec execute 242 sec latency 3326.916 ms 2024-08-16T19:17:43.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:43 smithi155 ceph-mon[37413]: pgmap v222: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-08-16T19:17:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:43 smithi062 ceph-mon[32993]: pgmap v222: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-08-16T19:17:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:43 smithi161 ceph-mon[37576]: pgmap v222: 129 pgs: 129 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-08-16T19:17:44.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 358972 4.72 MB/sec execute 243 sec latency 4326.970 ms 2024-08-16T19:17:45.940 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:45 smithi161 ceph-mon[37576]: pgmap v223: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-08-16T19:17:45.968 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359040 4.70 MB/sec execute 244 sec latency 4948.719 ms 2024-08-16T19:17:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:45 smithi062 ceph-mon[32993]: pgmap v223: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-08-16T19:17:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:45 smithi155 ceph-mon[37413]: pgmap v223: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-08-16T19:17:46.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359040 4.68 MB/sec execute 245 sec latency 1310.822 ms 2024-08-16T19:17:47.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359040 4.66 MB/sec execute 246 sec latency 2310.946 ms 2024-08-16T19:17:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:47 smithi062 ceph-mon[32993]: pgmap v224: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:48.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:47 smithi155 ceph-mon[37413]: pgmap v224: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:47 smithi161 ceph-mon[37576]: pgmap v224: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:48.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359040 4.65 MB/sec execute 247 sec latency 3310.987 ms 2024-08-16T19:17:49.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359040 4.63 MB/sec execute 248 sec latency 4311.109 ms 2024-08-16T19:17:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:49 smithi062 ceph-mon[32993]: pgmap v225: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:50.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:49 smithi155 ceph-mon[37413]: pgmap v225: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:50.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:49 smithi161 ceph-mon[37576]: pgmap v225: 129 pgs: 129 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:17:50.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 359768 4.63 MB/sec execute 249 sec latency 4933.632 ms 2024-08-16T19:17:50.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:50 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:51.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:50 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:51.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:50 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:17:51.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 361952 4.66 MB/sec execute 250 sec latency 81.592 ms 2024-08-16T19:17:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:51 smithi062 ceph-mon[32993]: pgmap v226: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-16T19:17:52.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:51 smithi155 ceph-mon[37413]: pgmap v226: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-16T19:17:52.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:51 smithi161 ceph-mon[37576]: pgmap v226: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-16T19:17:52.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 363201 4.67 MB/sec execute 251 sec latency 14.701 ms 2024-08-16T19:17:53.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 365319 4.70 MB/sec execute 252 sec latency 15.268 ms 2024-08-16T19:17:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:53 smithi062 ceph-mon[32993]: pgmap v227: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-08-16T19:17:54.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:53 smithi155 ceph-mon[37413]: pgmap v227: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-08-16T19:17:54.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:53 smithi161 ceph-mon[37576]: pgmap v227: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-08-16T19:17:54.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 366006 4.69 MB/sec execute 253 sec latency 320.442 ms 2024-08-16T19:17:55.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 366558 4.68 MB/sec execute 254 sec latency 938.987 ms 2024-08-16T19:17:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:55 smithi062 ceph-mon[32993]: pgmap v228: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-08-16T19:17:56.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:55 smithi155 ceph-mon[37413]: pgmap v228: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-08-16T19:17:56.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:55 smithi161 ceph-mon[37576]: pgmap v228: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-08-16T19:17:56.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 368642 4.71 MB/sec execute 255 sec latency 31.841 ms 2024-08-16T19:17:57.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 369606 4.71 MB/sec execute 256 sec latency 186.140 ms 2024-08-16T19:17:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:57 smithi062 ceph-mon[32993]: pgmap v229: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:17:58.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:57 smithi155 ceph-mon[37413]: pgmap v229: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:17:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:57 smithi161 ceph-mon[37576]: pgmap v229: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:17:58.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 369606 4.69 MB/sec execute 257 sec latency 1186.184 ms 2024-08-16T19:17:59.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 369606 4.67 MB/sec execute 258 sec latency 2186.290 ms 2024-08-16T19:17:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:17:59 smithi062 ceph-mon[32993]: pgmap v230: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:00.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:17:59 smithi155 ceph-mon[37413]: pgmap v230: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:00.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:17:59 smithi161 ceph-mon[37576]: pgmap v230: 129 pgs: 129 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:00.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 369913 4.66 MB/sec execute 259 sec latency 2554.033 ms 2024-08-16T19:18:01.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 371322 4.66 MB/sec execute 260 sec latency 61.714 ms 2024-08-16T19:18:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:01 smithi062 ceph-mon[32993]: pgmap v231: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-16T19:18:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:01 smithi155 ceph-mon[37413]: pgmap v231: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-16T19:18:02.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:01 smithi161 ceph-mon[37576]: pgmap v231: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-08-16T19:18:02.969 INFO:tasks.workunit.client.0.smithi062.stdout: 1 372966 4.69 MB/sec execute 261 sec latency 10.993 ms 2024-08-16T19:18:02.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:02 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:03.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:02 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:03.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:02 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:03.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 374809 4.70 MB/sec execute 262 sec latency 14.911 ms 2024-08-16T19:18:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:03 smithi062 ceph-mon[32993]: pgmap v232: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1015 KiB/s wr, 145 op/s 2024-08-16T19:18:04.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:03 smithi155 ceph-mon[37413]: pgmap v232: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1015 KiB/s wr, 145 op/s 2024-08-16T19:18:04.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:03 smithi161 ceph-mon[37576]: pgmap v232: 129 pgs: 129 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1015 KiB/s wr, 145 op/s 2024-08-16T19:18:04.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376409 4.73 MB/sec execute 263 sec latency 27.913 ms 2024-08-16T19:18:05.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376765 4.71 MB/sec execute 264 sec latency 386.563 ms 2024-08-16T19:18:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:05 smithi062 ceph-mon[32993]: pgmap v233: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-16T19:18:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:05 smithi062 ceph-mon[32993]: osdmap e96: 12 total, 12 up, 12 in 2024-08-16T19:18:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:05 smithi155 ceph-mon[37413]: pgmap v233: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-16T19:18:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:05 smithi155 ceph-mon[37413]: osdmap e96: 12 total, 12 up, 12 in 2024-08-16T19:18:06.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:05 smithi161 ceph-mon[37576]: pgmap v233: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-08-16T19:18:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:05 smithi161 ceph-mon[37576]: osdmap e96: 12 total, 12 up, 12 in 2024-08-16T19:18:06.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376765 4.69 MB/sec execute 265 sec latency 1264.250 ms 2024-08-16T19:18:07.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376765 4.67 MB/sec execute 266 sec latency 2264.376 ms 2024-08-16T19:18:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:07 smithi062 ceph-mon[32993]: pgmap v235: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:08.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:07 smithi155 ceph-mon[37413]: pgmap v235: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:08.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:07 smithi161 ceph-mon[37576]: pgmap v235: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:08.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376765 4.66 MB/sec execute 267 sec latency 3264.481 ms 2024-08-16T19:18:09.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 376765 4.64 MB/sec execute 268 sec latency 4264.558 ms 2024-08-16T19:18:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:09 smithi062 ceph-mon[32993]: pgmap v236: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:10.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:09 smithi155 ceph-mon[37413]: pgmap v236: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:10.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:09 smithi161 ceph-mon[37576]: pgmap v236: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-08-16T19:18:10.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 377467 4.64 MB/sec execute 269 sec latency 4887.977 ms 2024-08-16T19:18:11.670 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:18:11.671 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.7 2024-08-16T19:18:11.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:18:11.960 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:11.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 379770 4.67 MB/sec execute 270 sec latency 24.813 ms 2024-08-16T19:18:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:11 smithi062 ceph-mon[32993]: pgmap v237: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-08-16T19:18:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:11 smithi062 ceph-mon[32993]: osdmap e97: 12 total, 12 up, 12 in 2024-08-16T19:18:12.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:11 smithi155 ceph-mon[37413]: pgmap v237: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-08-16T19:18:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:11 smithi155 ceph-mon[37413]: osdmap e97: 12 total, 12 up, 12 in 2024-08-16T19:18:12.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:11 smithi161 ceph-mon[37576]: pgmap v237: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 198 op/s 2024-08-16T19:18:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:11 smithi161 ceph-mon[37576]: osdmap e97: 12 total, 12 up, 12 in 2024-08-16T19:18:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 -- 172.21.15.62:0/44294869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8102a60 msgr2=0x7fefb8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 --2- 172.21.15.62:0/44294869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8102a60 0x7fefb8102e40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009930 tx=0x7fefa802a420 comp rx=0 tx=0).stop 2024-08-16T19:18:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 -- 172.21.15.62:0/44294869 shutdown_connections 2024-08-16T19:18:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 --2- 172.21.15.62:0/44294869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8107fd0 0x7fefb810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 --2- 172.21.15.62:0/44294869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8103410 0x7fefb81078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 --2- 172.21.15.62:0/44294869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8102a60 0x7fefb8102e40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 -- 172.21.15.62:0/44294869 >> 172.21.15.62:0/44294869 conn(0x7fefb80fc500 msgr2=0x7fefb80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 -- 172.21.15.62:0/44294869 shutdown_connections 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.445+0000 7fefbc90a640 1 -- 172.21.15.62:0/44294869 wait complete. 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 Processor -- start 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 -- start start 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 0x7fefb8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8107fd0 0x7fefb819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:12.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8111dd0 con 0x7fefb8103410 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8111c50 con 0x7fefb8102a60 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefbc90a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8111f50 con 0x7fefb8107fd0 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 0x7fefb8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:60592/0 (socket says 172.21.15.62:60592) 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 -- 172.21.15.62:0/2542314161 learned_addr learned my addr 172.21.15.62:0/2542314161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb6d9d640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8107fd0 0x7fefb819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 -- 172.21.15.62:0/2542314161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8107fd0 msgr2=0x7fefb819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8107fd0 0x7fefb819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 -- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 msgr2=0x7fefb8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.446+0000 7fefb659c640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 0x7fefb8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefb659c640 1 -- 172.21.15.62:0/2542314161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb819d2c0 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefb5d9b640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 0x7fefb8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefb659c640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009230 tx=0x7fefa8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa803f030 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb819d550 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa8004440 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8035070 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.447+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb81005f0 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.448+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb806a580 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.449+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fefa8042030 con 0x7fefb8102a60 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.450+0000 7fefa77fe640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 0x7fefa00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.450+0000 7fefb5d9b640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 0x7fefa00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.450+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fefa80efb10 con 0x7fefb8102a60 2024-08-16T19:18:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.450+0000 7fefb5d9b640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 0x7fefa00a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fefb80feba0 tx=0x7fefac005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.453+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fefa80bbcf0 con 0x7fefb8102a60 2024-08-16T19:18:12.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.680+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefb8199670 con 0x7fefb8102a60 2024-08-16T19:18:12.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.681+0000 7fefa77fe640 1 -- 172.21.15.62:0/2542314161 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7fefa80bfff0 con 0x7fefb8102a60 2024-08-16T19:18:12.683 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:18:12.683 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":46,"btime":"2024-08-16T19:18:01:685216+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:18:00.847971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:18:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 46 2024-08-16T19:18:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 msgr2=0x7fefa00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 0x7fefa00a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fefb80feba0 tx=0x7fefac005f90 comp rx=0 tx=0).stop 2024-08-16T19:18:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 msgr2=0x7fefb8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009230 tx=0x7fefa8004040 comp rx=0 tx=0).stop 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 shutdown_connections 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fefa00a2cf0 0x7fefa00a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefb8107fd0 0x7fefb819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefb8103410 0x7fefb8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 --2- 172.21.15.62:0/2542314161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefb8102a60 0x7fefb8198310 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 >> 172.21.15.62:0/2542314161 conn(0x7fefb80fc500 msgr2=0x7fefb8105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.683+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 shutdown_connections 2024-08-16T19:18:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:12.684+0000 7fefbc90a640 1 -- 172.21.15.62:0/2542314161 wait complete. 2024-08-16T19:18:12.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380285 4.66 MB/sec execute 271 sec latency 400.298 ms 2024-08-16T19:18:12.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:12 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2542314161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:13.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:12 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2542314161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:13.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:12 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2542314161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:13.503 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.7 --await-for 45.0' 2024-08-16T19:18:13.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.7 --await-for 45.0 2024-08-16T19:18:13.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:13 smithi062 ceph-mon[32993]: pgmap v239: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-08-16T19:18:13.769 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:13.970 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380285 4.64 MB/sec execute 272 sec latency 1400.388 ms 2024-08-16T19:18:13.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:13 smithi155 ceph-mon[37413]: pgmap v239: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-08-16T19:18:14.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:13 smithi161 ceph-mon[37576]: pgmap v239: 129 pgs: 129 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-08-16T19:18:14.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 -- 172.21.15.62:0/2430285185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481018b0 msgr2=0x7f7248101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 --2- 172.21.15.62:0/2430285185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481018b0 0x7f7248101d10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7238009930 tx=0x7f723802a420 comp rx=0 tx=0).stop 2024-08-16T19:18:14.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 -- 172.21.15.62:0/2430285185 shutdown_connections 2024-08-16T19:18:14.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 --2- 172.21.15.62:0/2430285185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 0x7f724810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 --2- 172.21.15.62:0/2430285185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481018b0 0x7f7248101d10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 --2- 172.21.15.62:0/2430285185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481078b0 0x7f7248107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.353+0000 7f72503a4640 1 -- 172.21.15.62:0/2430285185 >> 172.21.15.62:0/2430285185 conn(0x7f72480fd5f0 msgr2=0x7f72480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- 172.21.15.62:0/2430285185 shutdown_connections 2024-08-16T19:18:14.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- 172.21.15.62:0/2430285185 wait complete. 2024-08-16T19:18:14.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 Processor -- start 2024-08-16T19:18:14.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- start start 2024-08-16T19:18:14.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481018b0 0x7f7248074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 0x7f7248074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7248111d50 con 0x7f72481078b0 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7248111bd0 con 0x7f72481018b0 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.354+0000 7f72503a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7248111ed0 con 0x7f7248102250 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42004/0 (socket says 172.21.15.62:42004) 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 -- 172.21.15.62:0/2036896143 learned_addr learned my addr 172.21.15.62:0/2036896143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724d93f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 0x7f7248074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e140640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481018b0 0x7f7248074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 -- 172.21.15.62:0/2036896143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 msgr2=0x7f7248074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 0x7f7248074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 -- 172.21.15.62:0/2036896143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481018b0 msgr2=0x7f7248074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481018b0 0x7f7248074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 -- 172.21.15.62:0/2036896143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72481a6810 con 0x7f72481078b0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f724e941640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f724000b970 tx=0x7f724000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f723f7fe640 1 -- 172.21.15.62:0/2036896143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7240019070 con 0x7f72481078b0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72481a6b00 con 0x7f72481078b0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f723f7fe640 1 -- 172.21.15.62:0/2036896143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7240014440 con 0x7f72481078b0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.355+0000 7f723f7fe640 1 -- 172.21.15.62:0/2036896143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72400075d0 con 0x7f72481078b0 2024-08-16T19:18:14.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.356+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72481a7010 con 0x7f72481078b0 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.358+0000 7f723f7fe640 1 -- 172.21.15.62:0/2036896143 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f724001c070 con 0x7f72481078b0 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 Processor -- start 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 -- start start 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eebf0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0a50 0x5650eebfadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650eeb55a70 con 0x5650eebfb310 2024-08-16T19:18:14.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650eeaed780 con 0x5650eebf0130 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f72527a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650eeb51a30 con 0x5650eebf0a50 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42012/0 (socket says 172.21.15.62:42012) 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f724d93f640 1 -- 172.21.15.62:0/3468829296 learned_addr learned my addr 172.21.15.62:0/3468829296 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f723f7fe640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 0x7f72340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.359+0000 7f724e140640 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0a50 0x5650eebfadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eebf0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724e941640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 0x7f72340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 -- 172.21.15.62:0/3468829296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0a50 msgr2=0x5650eebfadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0a50 0x5650eebfadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 -- 172.21.15.62:0/3468829296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 msgr2=0x5650eebf0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eebf0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 -- 172.21.15.62:0/3468829296 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ee8c7b90 con 0x5650eebfb310 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f723f7fe640 1 -- 172.21.15.62:0/2036896143 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f72400cec50 con 0x7f72481078b0 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724d93f640 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7238000c00 tx=0x7f7238005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860d5d8b16bd803a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f724e941640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 0x7f72340a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72400cc170 tx=0x7f7240078040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f723cdea640 1 -- 172.21.15.62:0/3468829296 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238044030 con 0x5650eebfb310 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f723cdea640 1 -- 172.21.15.62:0/3468829296 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7238005020 con 0x5650eebfb310 2024-08-16T19:18:14.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.360+0000 7f723cdea640 1 -- 172.21.15.62:0/3468829296 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238035070 con 0x5650eebfb310 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 -- 172.21.15.62:0/3468829296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 msgr2=0x5650eebfd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7238000c00 tx=0x7f7238005fd0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 -- 172.21.15.62:0/3468829296 shutdown_connections 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebfd700 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0a50 0x5650eebfadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 --2- 172.21.15.62:0/3468829296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eebf0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 -- 172.21.15.62:0/3468829296 >> 172.21.15.62:0/3468829296 conn(0x5650eebec6c0 msgr2=0x5650eebecaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 -- 172.21.15.62:0/3468829296 shutdown_connections 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.361+0000 7f72527a2740 1 -- 172.21.15.62:0/3468829296 wait complete. 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 -- start start 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 Processor -- start 2024-08-16T19:18:14.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0130 0x5650eebf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650eebfdee0 con 0x5650eebfb310 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650eeb10990 con 0x5650eebf0a50 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650eebfe060 con 0x5650eebf0130 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f72527a2740 10 client.0 _ll_register_callbacks cb 0x5650ee865090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42014/0 (socket says 172.21.15.62:42014) 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.362+0000 7f724d93f640 1 -- 172.21.15.62:0/1238742267 learned_addr learned my addr 172.21.15.62:0/1238742267 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0130 0x5650eebf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724e140640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 -- 172.21.15.62:0/1238742267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0130 msgr2=0x5650eebf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0130 0x5650eebf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 -- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 msgr2=0x5650eebf86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 -- 172.21.15.62:0/1238742267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650eeb0f6c0 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f724d93f640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7238031850 tx=0x7f7238040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cd8a85e94fb158 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f7231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238035070 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f7238004c00 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7238005020 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7238004e00 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f72527a2740 10 client.15876 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.363+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5650eec09b40 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7238005020 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7f723803d570 con 0x5650eebfb310 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 10 client.15876 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:14.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 10 client.15876 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 20 client.15876 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec0ae90 0x5650eec0d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 4 client.15876 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650eeb631a0 con 0x5650eec0ae90 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.364+0000 7f724e140640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec0ae90 0x5650eec0d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f724e140640 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec0ae90 0x5650eec0d280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bed9b65e935e4d9 server_cookie=f21a4c06708517da in_seq=0 out_seq=0 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f7231ffb640 10 client.15876 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f7231ffb640 1 -- 172.21.15.62:0/1238742267 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650eeb631a0 con 0x5650eec0ae90 2024-08-16T19:18:14.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f7231ffb640 10 client.15876 handle_command_reply: tid=0 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f72527a2740 1 client.15876 shutdown 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 msgr2=0x5650eebf8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7238031850 tx=0x7f7238040040 comp rx=0 tx=0).stop 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.365+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 shutdown_connections 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec0ae90 0x5650eec0d280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebfb310 0x5650eebf8c00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 --2- 172.21.15.62:0/1238742267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebf0130 0x5650eebf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 >> 172.21.15.62:0/1238742267 conn(0x5650eebec6c0 msgr2=0x5650eebf23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f7231ffb640 0 client.15876 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 shutdown_connections 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 1 -- 172.21.15.62:0/1238742267 wait complete. 2024-08-16T19:18:14.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.366+0000 7f72527a2740 20 client.15876 trim_cache size 0 max 16384 2024-08-16T19:18:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 Processor -- start 2024-08-16T19:18:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 -- start start 2024-08-16T19:18:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eebeaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b1150 con 0x5650eebf0130 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b2db0 con 0x5650eebf0a50 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f72527a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b1820 con 0x5650eebfb310 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f724d93f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42022/0 (socket says 172.21.15.62:42022) 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f724d93f640 1 -- 172.21.15.62:0/3234808126 learned_addr learned my addr 172.21.15.62:0/3234808126 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.383+0000 7f724e140640 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 -- 172.21.15.62:0/3234808126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 msgr2=0x5650eebeaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eebeaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 -- 172.21.15.62:0/3234808126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 msgr2=0x5650eebf5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 -- 172.21.15.62:0/3234808126 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650eebf66d0 con 0x5650eebf0130 2024-08-16T19:18:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724e140640 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:18:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f724d93f640 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72380096a0 tx=0x7f723802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380476aefa069858 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f7233fff640 1 -- 172.21.15.62:0/3234808126 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238035070 con 0x5650eebf0130 2024-08-16T19:18:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f7233fff640 1 -- 172.21.15.62:0/3234808126 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723805e020 con 0x5650eebf0130 2024-08-16T19:18:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.384+0000 7f7233fff640 1 -- 172.21.15.62:0/3234808126 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238044030 con 0x5650eebf0130 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- 172.21.15.62:0/3234808126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 msgr2=0x5650eebf53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72380096a0 tx=0x7f723802a420 comp rx=0 tx=0).stop 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- 172.21.15.62:0/3234808126 shutdown_connections 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eebeaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0a50 0x5650eebf5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- 172.21.15.62:0/3234808126 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0130 0x5650eebf53e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- 172.21.15.62:0/3234808126 >> 172.21.15.62:0/3234808126 conn(0x5650eebec6c0 msgr2=0x5650eebf3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- 172.21.15.62:0/3234808126 shutdown_connections 2024-08-16T19:18:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- 172.21.15.62:0/3234808126 wait complete. 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- start start 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 Processor -- start 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eec10710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eec0e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b2620 con 0x5650eebf0a50 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f72481b1f50 con 0x5650eebf0130 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f72481af150 con 0x5650eebfb310 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.385+0000 7f72527a2740 10 client.0 _ll_register_callbacks cb 0x7f72440093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42026/0 (socket says 172.21.15.62:42026) 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 -- 172.21.15.62:0/694024649 learned_addr learned my addr 172.21.15.62:0/694024649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:14.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724d93f640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eec0e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724d93f640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eec10710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 -- 172.21.15.62:0/694024649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 msgr2=0x5650eec0e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eec0e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 -- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 msgr2=0x5650eec10710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eec10710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 -- 172.21.15.62:0/694024649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650eec0e8f0 con 0x5650eebf0a50 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f724e140640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244018820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158e0397e6e429b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f7231ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244016430 con 0x5650eebf0a50 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f72440165d0 con 0x5650eebf0a50 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f723805e020 con 0x5650eebf0a50 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f724402d050 con 0x5650eebf0a50 2024-08-16T19:18:14.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f72527a2740 10 client.15882 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.386+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5650eec75890 con 0x5650eebf0a50 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723805e020 con 0x5650eebf0a50 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7f7244026860 con 0x5650eebf0a50 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 10 client.15882 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 10 client.15882 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 20 client.15882 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec76170 0x5650eec78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 4 client.15882 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.7, "await_for": 45.0}] 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.7, "await_for": 45.0}) -- 0x5650eeb5d4f0 con 0x5650eec76170 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.387+0000 7f724d93f640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec76170 0x5650eec78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.388+0000 7f724d93f640 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec76170 0x5650eec78590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f45b5e1363b669a server_cookie=76e5a7b963382a53 in_seq=0 out_seq=0 2024-08-16T19:18:14.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.388+0000 7f7231ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.453+0000 7f7231ffb640 1 -- 172.21.15.62:0/694024649 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650eeb5d4f0 con 0x5650eec76170 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.453+0000 7f7231ffb640 10 client.15882 handle_command_reply: tid=0 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 client.15882 shutdown 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 msgr2=0x5650eec0db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244018820 comp rx=0 tx=0).stop 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 shutdown_connections 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650eec76170 0x5650eec78590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650eebfb310 0x5650eec0e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650eebf0a50 0x5650eec0db00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 --2- 172.21.15.62:0/694024649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650eebf0130 0x5650eec10710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 >> 172.21.15.62:0/694024649 conn(0x5650eebec6c0 msgr2=0x5650eebf99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f7231ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 shutdown_connections 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 1 -- 172.21.15.62:0/694024649 wait complete. 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.454+0000 7f72527a2740 20 client.15882 trim_cache size 0 max 16384 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 46, 2024-08-16T19:18:14.456 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 30, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "9b0d05e1": { 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 30, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.7, 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:18:14.457 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:14.458 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 msgr2=0x7f72340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 0x7f72340a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72400cc170 tx=0x7f7240078040 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 msgr2=0x7f7248074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f724000b970 tx=0x7f724000bd70 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 shutdown_connections 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f72340a2d70 0x7f72340a5230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72481078b0 0x7f7248074ca0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7248102250 0x7f7248074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 --2- 172.21.15.62:0/2036896143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72481018b0 0x7f7248074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 >> 172.21.15.62:0/2036896143 conn(0x7f72480fd5f0 msgr2=0x7f72480fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 shutdown_connections 2024-08-16T19:18:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:14.455+0000 7f72503a4640 1 -- 172.21.15.62:0/2036896143 wait complete. 2024-08-16T19:18:14.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.62 MB/sec execute 273 sec latency 1879.142 ms 2024-08-16T19:18:15.164 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9b0d05e1', quiesce took 3.5 seconds. Will release after: 24.2 2024-08-16T19:18:15.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.61 MB/sec execute 274 sec latency 1521.769 ms 2024-08-16T19:18:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:15 smithi062 ceph-mon[32993]: pgmap v240: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:18:16.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:15 smithi155 ceph-mon[37413]: pgmap v240: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:18:16.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:15 smithi161 ceph-mon[37576]: pgmap v240: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-08-16T19:18:16.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.59 MB/sec execute 275 sec latency 2521.876 ms 2024-08-16T19:18:17.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.57 MB/sec execute 276 sec latency 3522.002 ms 2024-08-16T19:18:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:17 smithi062 ceph-mon[32993]: pgmap v241: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:17 smithi155 ceph-mon[37413]: pgmap v241: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:18.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:17 smithi161 ceph-mon[37576]: pgmap v241: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:18.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.56 MB/sec execute 277 sec latency 4522.065 ms 2024-08-16T19:18:19.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.54 MB/sec execute 278 sec latency 5522.187 ms 2024-08-16T19:18:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:19 smithi062 ceph-mon[32993]: pgmap v242: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:19 smithi155 ceph-mon[37413]: pgmap v242: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:20.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:19 smithi161 ceph-mon[37576]: pgmap v242: 129 pgs: 129 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-08-16T19:18:20.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.52 MB/sec execute 279 sec latency 6522.300 ms 2024-08-16T19:18:21.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.51 MB/sec execute 280 sec latency 7522.337 ms 2024-08-16T19:18:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:21 smithi062 ceph-mon[32993]: pgmap v243: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:22.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:21 smithi155 ceph-mon[37413]: pgmap v243: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:22.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:21 smithi161 ceph-mon[37576]: pgmap v243: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:22.971 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.49 MB/sec execute 281 sec latency 8522.385 ms 2024-08-16T19:18:23.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.48 MB/sec execute 282 sec latency 9522.433 ms 2024-08-16T19:18:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:23 smithi062 ceph-mon[32993]: pgmap v244: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1022 KiB/s wr, 79 op/s 2024-08-16T19:18:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:23 smithi155 ceph-mon[37413]: pgmap v244: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1022 KiB/s wr, 79 op/s 2024-08-16T19:18:24.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:23 smithi161 ceph-mon[37576]: pgmap v244: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1022 KiB/s wr, 79 op/s 2024-08-16T19:18:24.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.46 MB/sec execute 283 sec latency 10522.489 ms 2024-08-16T19:18:25.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.44 MB/sec execute 284 sec latency 11522.598 ms 2024-08-16T19:18:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:25 smithi062 ceph-mon[32993]: pgmap v245: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:26.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:25 smithi155 ceph-mon[37413]: pgmap v245: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:26.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:25 smithi161 ceph-mon[37576]: pgmap v245: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-08-16T19:18:26.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.43 MB/sec execute 285 sec latency 12522.646 ms 2024-08-16T19:18:27.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.41 MB/sec execute 286 sec latency 13522.749 ms 2024-08-16T19:18:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:27 smithi062 ceph-mon[32993]: pgmap v246: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:28.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:27 smithi155 ceph-mon[37413]: pgmap v246: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:28.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:27 smithi161 ceph-mon[37576]: pgmap v246: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:28.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.40 MB/sec execute 287 sec latency 14522.859 ms 2024-08-16T19:18:29.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.38 MB/sec execute 288 sec latency 15522.919 ms 2024-08-16T19:18:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:29 smithi062 ceph-mon[32993]: pgmap v247: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:29 smithi155 ceph-mon[37413]: pgmap v247: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:30.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:29 smithi161 ceph-mon[37576]: pgmap v247: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 343 KiB/s wr, 26 op/s 2024-08-16T19:18:30.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.37 MB/sec execute 289 sec latency 16523.022 ms 2024-08-16T19:18:31.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.35 MB/sec execute 290 sec latency 17523.146 ms 2024-08-16T19:18:32.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:31 smithi155 ceph-mon[37413]: pgmap v248: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 823 KiB/s wr, 46 op/s 2024-08-16T19:18:32.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:31 smithi161 ceph-mon[37576]: pgmap v248: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 823 KiB/s wr, 46 op/s 2024-08-16T19:18:32.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:31 smithi062 ceph-mon[32993]: pgmap v248: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 823 KiB/s wr, 46 op/s 2024-08-16T19:18:32.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.34 MB/sec execute 291 sec latency 18523.269 ms 2024-08-16T19:18:33.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.32 MB/sec execute 292 sec latency 19523.381 ms 2024-08-16T19:18:34.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:33 smithi155 ceph-mon[37413]: pgmap v249: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 709 KiB/s wr, 34 op/s 2024-08-16T19:18:34.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:33 smithi161 ceph-mon[37576]: pgmap v249: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 709 KiB/s wr, 34 op/s 2024-08-16T19:18:34.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:33 smithi062 ceph-mon[32993]: pgmap v249: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 709 KiB/s wr, 34 op/s 2024-08-16T19:18:34.972 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.31 MB/sec execute 293 sec latency 20523.431 ms 2024-08-16T19:18:35.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.29 MB/sec execute 294 sec latency 21523.514 ms 2024-08-16T19:18:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:35 smithi155 ceph-mon[37413]: pgmap v250: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 866 KiB/s wr, 54 op/s 2024-08-16T19:18:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:35 smithi161 ceph-mon[37576]: pgmap v250: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 866 KiB/s wr, 54 op/s 2024-08-16T19:18:36.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:35 smithi062 ceph-mon[32993]: pgmap v250: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 866 KiB/s wr, 54 op/s 2024-08-16T19:18:36.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.28 MB/sec execute 295 sec latency 22523.596 ms 2024-08-16T19:18:37.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.26 MB/sec execute 296 sec latency 23523.664 ms 2024-08-16T19:18:38.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:37 smithi155 ceph-mon[37413]: pgmap v251: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:37 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:18:38.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:37 smithi161 ceph-mon[37576]: pgmap v251: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:37 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:18:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:37 smithi062 ceph-mon[32993]: pgmap v251: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:37 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:18:38.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.25 MB/sec execute 297 sec latency 24523.745 ms 2024-08-16T19:18:39.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:18:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:18:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:18:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:18:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:18:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:18:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:18:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:18:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:18:39.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:18:39.645 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:39.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.24 MB/sec execute 298 sec latency 25523.781 ms 2024-08-16T19:18:40.064 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:39 smithi062 ceph-mon[32993]: pgmap v252: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:40.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:39 smithi155 ceph-mon[37413]: pgmap v252: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:40.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:39 smithi161 ceph-mon[37576]: pgmap v252: 129 pgs: 129 active+clean; 256 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 638 KiB/s wr, 39 op/s 2024-08-16T19:18:40.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 -- 172.21.15.62:0/1386658431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 msgr2=0x7fbca410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 --2- 172.21.15.62:0/1386658431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca410b880 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009900 tx=0x7fbc8c02a480 comp rx=0 tx=0).stop 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 -- 172.21.15.62:0/1386658431 shutdown_connections 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 --2- 172.21.15.62:0/1386658431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca410bea0 0x7fbca410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 --2- 172.21.15.62:0/1386658431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca410b880 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 --2- 172.21.15.62:0/1386658431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca4102a70 0x7fbca4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 -- 172.21.15.62:0/1386658431 >> 172.21.15.62:0/1386658431 conn(0x7fbca40fc4d0 msgr2=0x7fbca40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 -- 172.21.15.62:0/1386658431 shutdown_connections 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.147+0000 7fbcaa697640 1 -- 172.21.15.62:0/1386658431 wait complete. 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 Processor -- start 2024-08-16T19:18:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 -- start start 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 0x7fbca41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 0x7fbca419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111ca0 con 0x7fbca4103390 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111b20 con 0x7fbca410bea0 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbcaa697640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4111e20 con 0x7fbca4102a70 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbca37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42332/0 (socket says 172.21.15.62:42332) 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.148+0000 7fbca37fe640 1 -- 172.21.15.62:0/2043845530 learned_addr learned my addr 172.21.15.62:0/2043845530 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca3fff640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 0x7fbca41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca8c34640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 0x7fbca419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 -- 172.21.15.62:0/2043845530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 msgr2=0x7fbca41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 0x7fbca41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 -- 172.21.15.62:0/2043845530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 msgr2=0x7fbca419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 0x7fbca419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 -- 172.21.15.62:0/2043845530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca419d090 con 0x7fbca4103390 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca3fff640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 0x7fbca41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca8c34640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 0x7fbca419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca37fe640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005ab0 tx=0x7fbc8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c03f030 con 0x7fbca4103390 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca419d320 con 0x7fbca4103390 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c003ce0 con 0x7fbca4103390 2024-08-16T19:18:40.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c035070 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.149+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca41a4c60 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.150+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca4102e50 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.151+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc8c042030 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.152+0000 7fbca17fa640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 0x7fbc900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.152+0000 7fbca3fff640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 0x7fbc900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.152+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fbc8c0f03c0 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.153+0000 7fbca3fff640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 0x7fbc900a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400baa0 tx=0x7fbc94005d90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.155+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbc8c0bcbf0 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.367+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca4199440 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.367+0000 7fbca17fa640 1 -- 172.21.15.62:0/2043845530 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7fbca4199440 con 0x7fbca4103390 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 46 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.369+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 msgr2=0x7fbc900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.369+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 0x7fbc900a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc9400baa0 tx=0x7fbc94005d90 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.369+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 msgr2=0x7fbca4198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.369+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c005ab0 tx=0x7fbc8c0042f0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 shutdown_connections 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbc900a2cf0 0x7fbc900a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbca410bea0 0x7fbca419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca4103390 0x7fbca4198620 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 --2- 172.21.15.62:0/2043845530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbca4102a70 0x7fbca41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 >> 172.21.15.62:0/2043845530 conn(0x7fbca40fc4d0 msgr2=0x7fbca40fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 shutdown_connections 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:40.370+0000 7fbcaa697640 1 -- 172.21.15.62:0/2043845530 wait complete. 2024-08-16T19:18:40.481 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:18:40.482 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":46,"btime":"2024-08-16T19:18:01:685216+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:18:00.847971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:18:40.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.22 MB/sec execute 299 sec latency 26523.862 ms 2024-08-16T19:18:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:40 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2043845530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:40 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2043845530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:41.171 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:40 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2043845530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:41.203 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 9b0d05e1 --release --await' 2024-08-16T19:18:41.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 9b0d05e1 --release --await 2024-08-16T19:18:41.470 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:41.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 380286 4.21 MB/sec execute 300 sec latency 27523.955 ms 2024-08-16T19:18:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:41 smithi062 ceph-mon[32993]: pgmap v253: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-16T19:18:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:41 smithi155 ceph-mon[37413]: pgmap v253: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-16T19:18:42.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:41 smithi161 ceph-mon[37576]: pgmap v253: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.0 MiB/s wr, 57 op/s 2024-08-16T19:18:42.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 -- 172.21.15.62:0/2552340786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee381022a0 msgr2=0x7fee381026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/2552340786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee381022a0 0x7fee381026e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fee20009900 tx=0x7fee2002a480 comp rx=0 tx=0).stop 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 -- 172.21.15.62:0/2552340786 shutdown_connections 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/2552340786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38102c20 0x7fee3810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/2552340786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee381022a0 0x7fee381026e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/2552340786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee38100b50 0x7fee38100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 -- 172.21.15.62:0/2552340786 >> 172.21.15.62:0/2552340786 conn(0x7fee380fc500 msgr2=0x7fee380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 -- 172.21.15.62:0/2552340786 shutdown_connections 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 -- 172.21.15.62:0/2552340786 wait complete. 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.132+0000 7fee3eb6e640 1 Processor -- start 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 -- start start 2024-08-16T19:18:42.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38100b50 0x7fee38198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381022a0 0x7fee38198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee38110cd0 con 0x7fee38102c20 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee38110b50 con 0x7fee38100b50 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3eb6e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee38110e50 con 0x7fee381022a0 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60244/0 (socket says 172.21.15.62:60244) 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 -- 172.21.15.62:0/3056776372 learned_addr learned my addr 172.21.15.62:0/3056776372 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3c90a640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38100b50 0x7fee38198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee37fff640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381022a0 0x7fee38198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 -- 172.21.15.62:0/3056776372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381022a0 msgr2=0x7fee38198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381022a0 0x7fee38198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 -- 172.21.15.62:0/3056776372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38100b50 msgr2=0x7fee38198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38100b50 0x7fee38198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 -- 172.21.15.62:0/3056776372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3819d1d0 con 0x7fee38102c20 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee3d10b640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b970 tx=0x7fee2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.133+0000 7fee35ffb640 1 -- 172.21.15.62:0/3056776372 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28019070 con 0x7fee38102c20 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.134+0000 7fee35ffb640 1 -- 172.21.15.62:0/3056776372 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2800cb40 con 0x7fee38102c20 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.134+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3819d4c0 con 0x7fee38102c20 2024-08-16T19:18:42.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.134+0000 7fee35ffb640 1 -- 172.21.15.62:0/3056776372 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28007650 con 0x7fee38102c20 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.134+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381a4da0 con 0x7fee38102c20 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.136+0000 7fee35ffb640 1 -- 172.21.15.62:0/3056776372 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee2801c070 con 0x7fee38102c20 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.136+0000 7fee40f6c740 1 Processor -- start 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.136+0000 7fee40f6c740 1 -- start start 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a9653cc60 con 0x558a965d6890 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558a964d4b90 con 0x558a965d71b0 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee40f6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558a96538c20 con 0x558a965e5f80 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee35ffb640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 0x7fee240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60260/0 (socket says 172.21.15.62:60260) 2024-08-16T19:18:42.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 -- 172.21.15.62:0/1922224257 learned_addr learned my addr 172.21.15.62:0/1922224257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:42.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee3d10b640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 0x7fee240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee3c90a640 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee35ffb640 1 -- 172.21.15.62:0/3056776372 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fee280cedd0 con 0x7fee38102c20 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 -- 172.21.15.62:0/1922224257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 msgr2=0x558a965e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 -- 172.21.15.62:0/1922224257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 msgr2=0x558a965e5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965e5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 -- 172.21.15.62:0/1922224257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9624c580 con 0x558a965d6890 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.137+0000 7fee37fff640 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee20005940 tx=0x7fee20005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3388cb33e5c5202 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee3d10b640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 0x7fee240a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee2809afa0 tx=0x7fee28096c10 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee2effd640 1 -- 172.21.15.62:0/1922224257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20045070 con 0x558a965d6890 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee2effd640 1 -- 172.21.15.62:0/1922224257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee200046c0 con 0x558a965d6890 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee2effd640 1 -- 172.21.15.62:0/1922224257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20035070 con 0x558a965d6890 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 -- 172.21.15.62:0/1922224257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 msgr2=0x558a965d6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee20005940 tx=0x7fee20005a20 comp rx=0 tx=0).stop 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 -- 172.21.15.62:0/1922224257 shutdown_connections 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965e5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 --2- 172.21.15.62:0/1922224257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965d6c70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 -- 172.21.15.62:0/1922224257 >> 172.21.15.62:0/1922224257 conn(0x558a965d2f20 msgr2=0x558a965d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 -- 172.21.15.62:0/1922224257 shutdown_connections 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.138+0000 7fee40f6c740 1 -- 172.21.15.62:0/1922224257 wait complete. 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 -- start start 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 Processor -- start 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d71b0 0x558a965e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558a965e8b50 con 0x558a965d6890 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x558a964da7c0 con 0x558a965e5f80 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558a965e8cd0 con 0x558a965d71b0 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee40f6c740 10 client.0 _ll_register_callbacks cb 0x558a964e3f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60276/0 (socket says 172.21.15.62:60276) 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.139+0000 7fee37fff640 1 -- 172.21.15.62:0/3190173155 learned_addr learned my addr 172.21.15.62:0/3190173155 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee3c90a640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d71b0 0x558a965e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 -- 172.21.15.62:0/3190173155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d71b0 msgr2=0x558a965e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d71b0 0x558a965e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 -- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 msgr2=0x558a965e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 -- 172.21.15.62:0/3190173155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a96316960 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee37fff640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fee20005910 tx=0x7fee200095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299adfaec73ba1ea server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee17fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20035070 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee200046c0 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fee20004d20 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fee20055050 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee40f6c740 10 client.15900 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558a965f4c50 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.140+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee200046c0 con 0x558a965d6890 2024-08-16T19:18:42.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7fee2004b030 con 0x558a965d6890 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 10 client.15900 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 10 client.15900 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 20 client.15900 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a965f5fa0 0x558a965f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 4 client.15900 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a9654a1d0 con 0x558a965f5fa0 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee3c90a640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a965f5fa0 0x558a965f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee17fff640 10 client.15900 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.141+0000 7fee3c90a640 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a965f5fa0 0x558a965f8390 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4ef3eb13a78a89 server_cookie=d5dac0ecc1f8775c in_seq=0 out_seq=0 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee17fff640 1 -- 172.21.15.62:0/3190173155 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a9654a1d0 con 0x558a965f5fa0 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee17fff640 10 client.15900 handle_command_reply: tid=0 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 client.15900 shutdown 2024-08-16T19:18:42.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 msgr2=0x558a965e2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fee20005910 tx=0x7fee200095f0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 shutdown_connections 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a965f5fa0 0x558a965f8390 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d71b0 0x558a965e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 --2- 172.21.15.62:0/3190173155 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965e2ee0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 >> 172.21.15.62:0/3190173155 conn(0x558a965d2f20 msgr2=0x558a965d8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.142+0000 7fee17fff640 0 client.15900 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.143+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 shutdown_connections 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.143+0000 7fee40f6c740 1 -- 172.21.15.62:0/3190173155 wait complete. 2024-08-16T19:18:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.143+0000 7fee40f6c740 20 client.15900 trim_cache size 0 max 16384 2024-08-16T19:18:42.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 Processor -- start 2024-08-16T19:18:42.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 -- start start 2024-08-16T19:18:42.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d6890 0x558a965d8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee381acc40 con 0x558a965d71b0 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee381ae8a0 con 0x558a965e5f80 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee40f6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee381ad310 con 0x558a965d6890 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60280/0 (socket says 172.21.15.62:60280) 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 -- 172.21.15.62:0/135820118 learned_addr learned my addr 172.21.15.62:0/135820118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee37fff640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d6890 0x558a965d8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee37fff640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 -- 172.21.15.62:0/135820118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d6890 msgr2=0x558a965d8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d6890 0x558a965d8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 -- 172.21.15.62:0/135820118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 msgr2=0x558a965fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee3c90a640 1 -- 172.21.15.62:0/135820118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a965deb20 con 0x558a965d71b0 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.160+0000 7fee37fff640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee3c90a640 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x558a965e2770 tx=0x7fee3000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da813eedcc936b6b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee2e7fc640 1 -- 172.21.15.62:0/135820118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30003aa0 con 0x558a965d71b0 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee2e7fc640 1 -- 172.21.15.62:0/135820118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee30005500 con 0x558a965d71b0 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee2e7fc640 1 -- 172.21.15.62:0/135820118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30010430 con 0x558a965d71b0 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 -- 172.21.15.62:0/135820118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 msgr2=0x558a965fafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x558a965e2770 tx=0x7fee3000cba0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 -- 172.21.15.62:0/135820118 shutdown_connections 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965e5f80 0x558a965fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d71b0 0x558a965fafd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 --2- 172.21.15.62:0/135820118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965d6890 0x558a965d8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.161+0000 7fee40f6c740 1 -- 172.21.15.62:0/135820118 >> 172.21.15.62:0/135820118 conn(0x558a965d2f20 msgr2=0x558a965d8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- 172.21.15.62:0/135820118 shutdown_connections 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- 172.21.15.62:0/135820118 wait complete. 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- start start 2024-08-16T19:18:42.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 Processor -- start 2024-08-16T19:18:42.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fee381ae110 con 0x558a965d6890 2024-08-16T19:18:42.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fee381ada40 con 0x558a965d71b0 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee381aac40 con 0x558a965e5f80 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee40f6c740 10 client.0 _ll_register_callbacks cb 0x7fee200044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.162+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60282/0 (socket says 172.21.15.62:60282) 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 -- 172.21.15.62:0/2417810513 learned_addr learned my addr 172.21.15.62:0/2417810513 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee3c90a640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 -- 172.21.15.62:0/2417810513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 msgr2=0x558a965f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 -- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 msgr2=0x558a965f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 -- 172.21.15.62:0/2417810513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9665fa80 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee3c90a640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee37fff640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fee2002a4c0 tx=0x7fee20005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c0acc3d81af611 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee17fff640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20060020 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fee20004a80 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee30005500 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fee20068050 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee40f6c740 10 client.15906 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.163+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558a96660d10 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee30005500 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7fee2004b040 con 0x558a965d6890 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 10 client.15906 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 10 client.15906 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:42.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 20 client.15906 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a966613d0 0x558a966637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 4 client.15906 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b0d05e1", "release": true, "await": true}] 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b0d05e1", "release": true, "await": true}) -- 0x558a965ce1d0 con 0x558a966613d0 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.164+0000 7fee3c90a640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a966613d0 0x558a966637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.165+0000 7fee17fff640 10 client.15906 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:42.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.165+0000 7fee3c90a640 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a966613d0 0x558a966637f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a91b03edb843bb server_cookie=cea5155ab5a8e01d in_seq=0 out_seq=0 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee17fff640 1 -- 172.21.15.62:0/2417810513 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558a965ce1d0 con 0x558a966613d0 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee17fff640 10 client.15906 handle_command_reply: tid=0 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 client.15906 shutdown 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 msgr2=0x558a965f8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fee2002a4c0 tx=0x7fee20005490 comp rx=0 tx=0).stop 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 shutdown_connections 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x558a966613d0 0x558a966637f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558a965e5f80 0x558a965f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a965d71b0 0x558a965f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 --2- 172.21.15.62:0/2417810513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558a965d6890 0x558a965f8f40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 >> 172.21.15.62:0/2417810513 conn(0x558a965d2f20 msgr2=0x558a965f1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee17fff640 0 client.15906 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 shutdown_connections 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 1 -- 172.21.15.62:0/2417810513 wait complete. 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.172+0000 7fee40f6c740 20 client.15906 trim_cache size 0 max 16384 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 46, 2024-08-16T19:18:42.174 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 32, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "9b0d05e1": { 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 32, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:18:42.175 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.7, 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:42.176 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:42.177 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:18:42.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 msgr2=0x7fee240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 0x7fee240a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee2809afa0 tx=0x7fee28096c10 comp rx=0 tx=0).stop 2024-08-16T19:18:42.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 msgr2=0x7fee3819caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:42.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee2800b970 tx=0x7fee2800be30 comp rx=0 tx=0).stop 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 shutdown_connections 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fee240a2cf0 0x7fee240a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fee38102c20 0x7fee3819caf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee381022a0 0x7fee38198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 --2- 172.21.15.62:0/3056776372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee38100b50 0x7fee38198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 >> 172.21.15.62:0/3056776372 conn(0x7fee380fc500 msgr2=0x7fee38069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 shutdown_connections 2024-08-16T19:18:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:42.173+0000 7fee3eb6e640 1 -- 172.21.15.62:0/3056776372 wait complete. 2024-08-16T19:18:42.861 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9b0d05e1', total seconds elapsed: 31.2 2024-08-16T19:18:42.862 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.7 2024-08-16T19:18:42.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:18:42.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 381959 4.22 MB/sec execute 301 sec latency 27725.297 ms 2024-08-16T19:18:43.152 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 -- 172.21.15.62:0/3312019038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 msgr2=0x7f475c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 --2- 172.21.15.62:0/3312019038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c10e780 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4754009cd0 tx=0x7f475402a8e0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 -- 172.21.15.62:0/3312019038 shutdown_connections 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 --2- 172.21.15.62:0/3312019038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c10e780 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 --2- 172.21.15.62:0/3312019038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1018b0 0x7f475c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 --2- 172.21.15.62:0/3312019038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1078b0 0x7f475c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 -- 172.21.15.62:0/3312019038 >> 172.21.15.62:0/3312019038 conn(0x7f475c0fd5f0 msgr2=0x7f475c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 -- 172.21.15.62:0/3312019038 shutdown_connections 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.625+0000 7f476340f640 1 -- 172.21.15.62:0/3312019038 wait complete. 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 Processor -- start 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 -- start start 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1018b0 0x7f475c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 0x7f475c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:43.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111cb0 con 0x7f475c102250 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111b30 con 0x7f475c1078b0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.626+0000 7f476340f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f475c111e30 con 0x7f475c1018b0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47609aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 0x7f475c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47609aa640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60306/0 (socket says 172.21.15.62:60306) 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47609aa640 1 -- 172.21.15.62:0/2125996285 learned_addr learned my addr 172.21.15.62:0/2125996285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47611ab640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1018b0 0x7f475c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 -- 172.21.15.62:0/2125996285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1018b0 msgr2=0x7f475c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1018b0 0x7f475c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 -- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 msgr2=0x7f475c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 -- 172.21.15.62:0/2125996285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c1a6620 con 0x7f475c1078b0 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47609aa640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:18:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47619ac640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 0x7f475c074ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f475402ae00 tx=0x7f4754031df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754003d80 con 0x7f475c1078b0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4754004520 con 0x7f475c1078b0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.627+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c1a68b0 con 0x7f475c1078b0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.628+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754030d00 con 0x7f475c1078b0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.628+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c1a6df0 con 0x7f475c1078b0 2024-08-16T19:18:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.629+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475c101d10 con 0x7f475c1078b0 2024-08-16T19:18:43.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.630+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47540046d0 con 0x7f475c1078b0 2024-08-16T19:18:43.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.631+0000 7f47527fc640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 0x7f47480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:43.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.631+0000 7f47611ab640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 0x7f47480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:43.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.631+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f47540f07c0 con 0x7f475c1078b0 2024-08-16T19:18:43.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.631+0000 7f47611ab640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 0x7f47480a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4758009b50 tx=0x7f4758005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:43.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.634+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f47540bd0a0 con 0x7f475c1078b0 2024-08-16T19:18:43.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.865+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f475c02d5e0 con 0x7f475c1078b0 2024-08-16T19:18:43.866 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:43 smithi062 ceph-mon[32993]: pgmap v254: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 591 KiB/s wr, 37 op/s 2024-08-16T19:18:43.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.866+0000 7f47527fc640 1 -- 172.21.15.62:0/2125996285 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11999 (secure 0 0 0) 0x7f47540bc9a0 con 0x7f475c1078b0 2024-08-16T19:18:43.867 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:18:43.867 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":46,"btime":"2024-08-16T19:18:01:685216+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:18:00.847971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:18:43.867 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 46 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 msgr2=0x7f47480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 0x7f47480a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4758009b50 tx=0x7f4758005f90 comp rx=0 tx=0).stop 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 msgr2=0x7f475c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 0x7f475c074ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f475402ae00 tx=0x7f4754031df0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 shutdown_connections 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f47480a2cf0 0x7f47480a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f475c1078b0 0x7f475c074ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f475c102250 0x7f475c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 --2- 172.21.15.62:0/2125996285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475c1018b0 0x7f475c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 >> 172.21.15.62:0/2125996285 conn(0x7f475c0fd5f0 msgr2=0x7f475c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 shutdown_connections 2024-08-16T19:18:43.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:43.869+0000 7f476340f640 1 -- 172.21.15.62:0/2125996285 wait complete. 2024-08-16T19:18:43.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 383570 4.24 MB/sec execute 302 sec latency 11.335 ms 2024-08-16T19:18:44.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:43 smithi155 ceph-mon[37413]: pgmap v254: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 591 KiB/s wr, 37 op/s 2024-08-16T19:18:44.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:43 smithi161 ceph-mon[37576]: pgmap v254: 129 pgs: 129 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 591 KiB/s wr, 37 op/s 2024-08-16T19:18:44.542 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.7 --await-for 45.0' 2024-08-16T19:18:44.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.7 --await-for 45.0 2024-08-16T19:18:44.840 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:18:44.853 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:44 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2125996285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:44.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 385498 4.25 MB/sec execute 303 sec latency 14.798 ms 2024-08-16T19:18:45.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:44 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2125996285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:45.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:44 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2125996285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.340+0000 7f13b741e640 1 -- 172.21.15.62:0/2972055820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 msgr2=0x7f13b010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.340+0000 7f13b741e640 1 --2- 172.21.15.62:0/2972055820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b010d770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009ca0 tx=0x7f13a802a950 comp rx=0 tx=0).stop 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 -- 172.21.15.62:0/2972055820 shutdown_connections 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 --2- 172.21.15.62:0/2972055820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b010d770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 --2- 172.21.15.62:0/2972055820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b01022a0 0x7f13b01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 --2- 172.21.15.62:0/2972055820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b0100b50 0x7f13b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 -- 172.21.15.62:0/2972055820 >> 172.21.15.62:0/2972055820 conn(0x7f13b00fc500 msgr2=0x7f13b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 -- 172.21.15.62:0/2972055820 shutdown_connections 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 -- 172.21.15.62:0/2972055820 wait complete. 2024-08-16T19:18:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 Processor -- start 2024-08-16T19:18:45.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.341+0000 7f13b741e640 1 -- start start 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 0x7f13b019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010fbf0 con 0x7f13b0102c20 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010fa70 con 0x7f13b01022a0 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b741e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13b010fd70 con 0x7f13b0100b50 2024-08-16T19:18:45.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:40188/0 (socket says 172.21.15.62:40188) 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b49b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 0x7f13b019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 -- 172.21.15.62:0/4121218154 learned_addr learned my addr 172.21.15.62:0/4121218154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b59bb640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 -- 172.21.15.62:0/4121218154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 msgr2=0x7f13b019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 0x7f13b019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 -- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 msgr2=0x7f13b019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b51ba640 1 -- 172.21.15.62:0/4121218154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b019f3a0 con 0x7f13b0100b50 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.342+0000 7f13b49b9640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 0x7f13b019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13b51ba640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9f0 tx=0x7f13ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13a27fc640 1 -- 172.21.15.62:0/4121218154 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00cd80 con 0x7f13b0100b50 2024-08-16T19:18:45.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13a27fc640 1 -- 172.21.15.62:0/4121218154 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac010910 con 0x7f13b0100b50 2024-08-16T19:18:45.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b019f690 con 0x7f13b0100b50 2024-08-16T19:18:45.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13a27fc640 1 -- 172.21.15.62:0/4121218154 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac018990 con 0x7f13b0100b50 2024-08-16T19:18:45.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.343+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b01a6f70 con 0x7f13b0100b50 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13b981c740 1 Processor -- start 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13a27fc640 1 -- 172.21.15.62:0/4121218154 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13ac010ac0 con 0x7f13b0100b50 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13b981c740 1 -- start start 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13b981c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 0x564308013510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13b981c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.346+0000 7f13b981c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 0x564308020700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b981c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564307f78a70 con 0x564308013a50 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b981c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564307f10780 con 0x56430801e310 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b981c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564307f74a30 con 0x564308013130 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 0x564308013510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 0x564308020700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 0x564308013510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:40204/0 (socket says 172.21.15.62:40204) 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b59bb640 1 -- 172.21.15.62:0/3445766462 learned_addr learned my addr 172.21.15.62:0/3445766462 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:45.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b51ba640 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 -- 172.21.15.62:0/3445766462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 msgr2=0x564308013510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 0x564308013510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 -- 172.21.15.62:0/3445766462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 msgr2=0x56430801ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 -- 172.21.15.62:0/3445766462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564307ceab90 con 0x56430801e310 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13a27fc640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 0x7f139c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b49b9640 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 0x564308020700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009f20 tx=0x7f13a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af009f85e7d26f6f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13b59bb640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 0x7f139c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.347+0000 7f13a27fc640 1 -- 172.21.15.62:0/4121218154 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f13ac0cfbb0 con 0x7f13b0100b50 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f139b7fe640 1 -- 172.21.15.62:0/3445766462 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004030 con 0x56430801e310 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f139b7fe640 1 -- 172.21.15.62:0/3445766462 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a4005500 con 0x56430801e310 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f139b7fe640 1 -- 172.21.15.62:0/3445766462 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004dd0 con 0x56430801e310 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f13b981c740 1 -- 172.21.15.62:0/3445766462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 msgr2=0x564308020700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f13b59bb640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 0x7f139c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009a60 tx=0x7f13a8005d80 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.348+0000 7f13b981c740 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 0x564308020700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009f20 tx=0x7f13a40118d0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 -- 172.21.15.62:0/3445766462 shutdown_connections 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430801e310 0x564308020700 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009f20 tx=0x7f13a40118d0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 --2- 172.21.15.62:0/3445766462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564308013130 0x564308013510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 -- 172.21.15.62:0/3445766462 >> 172.21.15.62:0/3445766462 conn(0x56430800f6c0 msgr2=0x56430800faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 -- 172.21.15.62:0/3445766462 shutdown_connections 2024-08-16T19:18:45.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 -- 172.21.15.62:0/3445766462 wait complete. 2024-08-16T19:18:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 -- start start 2024-08-16T19:18:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.349+0000 7f13b981c740 1 Processor -- start 2024-08-16T19:18:45.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013130 0x56430801b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x564308020ee0 con 0x564308013a50 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x564307f33990 con 0x564308013130 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564308021060 con 0x56430801be40 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b981c740 10 client.0 _ll_register_callbacks cb 0x564307c88090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:40216/0 (socket says 172.21.15.62:40216) 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b59bb640 1 -- 172.21.15.62:0/418153266 learned_addr learned my addr 172.21.15.62:0/418153266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b49b9640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.350+0000 7f13b51ba640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013130 0x56430801b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 -- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013130 msgr2=0x56430801b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013130 0x56430801b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 -- 172.21.15.62:0/418153266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 msgr2=0x56430801b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 -- 172.21.15.62:0/418153266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564307f326c0 con 0x56430801be40 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b49b9640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b59bb640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13a8003cd0 tx=0x7f13a80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28bd4758a2744fb server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f1398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8055060 con 0x56430801be40 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f13a8003f40 con 0x56430801be40 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.351+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a4005500 con 0x56430801be40 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f13a805c050 con 0x56430801be40 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 10 client.25519 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:45.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56430802d180 con 0x56430801be40 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13a4005500 con 0x56430801be40 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7f13a8030a10 con 0x56430801be40 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 10 client.25519 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 10 client.25519 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 20 client.25519 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56430802e690 0x564308030b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 4 client.25519 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564307f861a0 con 0x56430802e690 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.352+0000 7f13b51ba640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56430802e690 0x564308030b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.353+0000 7f1398ff9640 10 client.25519 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.353+0000 7f13b51ba640 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56430802e690 0x564308030b10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36488d66d1554fd server_cookie=87b41f0865f4a8b3 in_seq=0 out_seq=0 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.353+0000 7f1398ff9640 1 -- 172.21.15.62:0/418153266 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564307f861a0 con 0x56430802e690 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.353+0000 7f1398ff9640 10 client.25519 handle_command_reply: tid=0 2024-08-16T19:18:45.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.353+0000 7f13b981c740 1 client.25519 shutdown 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 msgr2=0x56430802a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13a8003cd0 tx=0x7f13a80095c0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 shutdown_connections 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x56430802e690 0x564308030b10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x56430802a350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013a50 0x56430801b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 --2- 172.21.15.62:0/418153266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013130 0x56430801b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 >> 172.21.15.62:0/418153266 conn(0x56430800f6c0 msgr2=0x564308015420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f1398ff9640 0 client.25519 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 shutdown_connections 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 1 -- 172.21.15.62:0/418153266 wait complete. 2024-08-16T19:18:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.354+0000 7f13b981c740 20 client.25519 trim_cache size 0 max 16384 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 Processor -- start 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 -- start start 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308036920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01b1020 con 0x564308013130 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01b2c80 con 0x564308013a50 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b981c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01b16f0 con 0x56430801be40 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60354/0 (socket says 172.21.15.62:60354) 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 -- 172.21.15.62:0/3183403002 learned_addr learned my addr 172.21.15.62:0/3183403002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:45.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b59bb640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308036920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b49b9640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 -- 172.21.15.62:0/3183403002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 msgr2=0x564308036920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308036920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 -- 172.21.15.62:0/3183403002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 msgr2=0x5643080363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b51ba640 1 -- 172.21.15.62:0/3183403002 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564308009900 con 0x564308013130 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.373+0000 7f13b49b9640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.374+0000 7f13b51ba640 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9c0 tx=0x7f13ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed6bd046ccca425 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.374+0000 7f139affd640 1 -- 172.21.15.62:0/3183403002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0a48f0 con 0x564308013130 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.374+0000 7f139affd640 1 -- 172.21.15.62:0/3183403002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac09a030 con 0x564308013130 2024-08-16T19:18:45.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.374+0000 7f139affd640 1 -- 172.21.15.62:0/3183403002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac018e20 con 0x564308013130 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- 172.21.15.62:0/3183403002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 msgr2=0x56430802a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9c0 tx=0x7f13ac000bc0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- 172.21.15.62:0/3183403002 shutdown_connections 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308036920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 --2- 172.21.15.62:0/3183403002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x56430802a140 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- 172.21.15.62:0/3183403002 >> 172.21.15.62:0/3183403002 conn(0x56430800f6c0 msgr2=0x56430801b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- 172.21.15.62:0/3183403002 shutdown_connections 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- 172.21.15.62:0/3183403002 wait complete. 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 -- start start 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.375+0000 7f13b981c740 1 Processor -- start 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x564308018480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01b24f0 con 0x564308013130 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01b1e20 con 0x564308013a50 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01af020 con 0x56430801be40 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b981c740 10 client.0 _ll_register_callbacks cb 0x7f13a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b51ba640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x564308018480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:40236/0 (socket says 172.21.15.62:40236) 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 -- 172.21.15.62:0/903812332 learned_addr learned my addr 172.21.15.62:0/903812332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b49b9640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 -- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 msgr2=0x5643080189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 -- 172.21.15.62:0/903812332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 msgr2=0x564308018480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x564308018480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.376+0000 7f13b59bb640 1 -- 172.21.15.62:0/903812332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643080a0c70 con 0x56430801be40 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f13b59bb640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13a8030e30 tx=0x7f13a802b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3408d0998d80a6d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f1398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8065060 con 0x56430801be40 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac09a030 con 0x56430801be40 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f13a803a8a0 con 0x56430801be40 2024-08-16T19:18:45.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f13a806c050 con 0x56430801be40 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f13b981c740 10 client.25525 fetch_fsmap learned FSMap version 46 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5643080a1ea0 con 0x56430801be40 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.377+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac09a030 con 0x56430801be40 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) ==== 8394+0+0 (secure 0 0 0) 0x7f13a8003f40 con 0x56430801be40 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 10 client.25525 fetch_fsmap finished waiting for FSMap version 46 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 10 client.25525 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 20 client.25525 populate_metadata read hostname 'smithi062' 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5643080a26f0 0x5643080a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 4 client.25525 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.7, "await_for": 45.0}] 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.7, "await_for": 45.0}) -- 0x564307f804f0 con 0x5643080a26f0 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b51ba640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5643080a26f0 0x5643080a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f1398ff9640 10 client.25525 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:45.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.378+0000 7f13b51ba640 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5643080a26f0 0x5643080a4b10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71cd6c4d03ceb3f server_cookie=64a833372ad5a264 in_seq=0 out_seq=0 2024-08-16T19:18:45.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f1398ff9640 1 -- 172.21.15.62:0/903812332 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564307f804f0 con 0x5643080a26f0 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f1398ff9640 10 client.25525 handle_command_reply: tid=0 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 client.25525 shutdown 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 msgr2=0x564308018f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13a8030e30 tx=0x7f13a802b0d0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 shutdown_connections 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5643080a26f0 0x5643080a4b10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56430801be40 0x564308018f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564308013a50 0x5643080189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f1398ff9640 0 client.25525 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:18:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 --2- 172.21.15.62:0/903812332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564308013130 0x564308018480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.441+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 >> 172.21.15.62:0/903812332 conn(0x56430800f6c0 msgr2=0x56430800ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 shutdown_connections 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b981c740 1 -- 172.21.15.62:0/903812332 wait complete. 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b981c740 20 client.25525 trim_cache size 0 max 16384 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 46, 2024-08-16T19:18:45.444 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 34, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "b43a572b": { 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 34, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:18:45.445 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.7, 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:45.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:45.447 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:18:45.447 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:18:45.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 msgr2=0x7f139c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 0x7f139c0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009a60 tx=0x7f13a8005d80 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 msgr2=0x7f13b019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9f0 tx=0x7f13ac00eeb0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 shutdown_connections 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f139c0a2cf0 0x7f139c0a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13b0102c20 0x7f13b019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b01022a0 0x7f13b019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 --2- 172.21.15.62:0/4121218154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13b0100b50 0x7f13b019a3f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.442+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 >> 172.21.15.62:0/4121218154 conn(0x7f13b00fc500 msgr2=0x7f13b01042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.443+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 shutdown_connections 2024-08-16T19:18:45.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:18:45.443+0000 7f13b741e640 1 -- 172.21.15.62:0/4121218154 wait complete. 2024-08-16T19:18:45.973 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.26 MB/sec execute 304 sec latency 573.403 ms 2024-08-16T19:18:46.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:45 smithi155 ceph-mon[37413]: pgmap v255: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-16T19:18:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:45 smithi161 ceph-mon[37576]: pgmap v255: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-16T19:18:46.155 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:45 smithi062 ceph-mon[32993]: pgmap v255: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-16T19:18:46.193 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b43a572b', quiesce took 3.3 seconds. Will release after: 24.4 2024-08-16T19:18:46.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.24 MB/sec execute 305 sec latency 1573.524 ms 2024-08-16T19:18:47.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:46 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:46 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:47.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:46 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:18:47.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.23 MB/sec execute 306 sec latency 2573.585 ms 2024-08-16T19:18:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:47 smithi155 ceph-mon[37413]: pgmap v256: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:47 smithi161 ceph-mon[37576]: pgmap v256: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:48.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:47 smithi062 ceph-mon[32993]: pgmap v256: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:48.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.22 MB/sec execute 307 sec latency 3573.635 ms 2024-08-16T19:18:49.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.20 MB/sec execute 308 sec latency 4573.713 ms 2024-08-16T19:18:50.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:49 smithi155 ceph-mon[37413]: pgmap v257: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:49 smithi161 ceph-mon[37576]: pgmap v257: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:50.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:49 smithi062 ceph-mon[32993]: pgmap v257: 129 pgs: 129 active+clean; 258 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-08-16T19:18:50.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.19 MB/sec execute 309 sec latency 5573.759 ms 2024-08-16T19:18:51.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.18 MB/sec execute 310 sec latency 6573.888 ms 2024-08-16T19:18:52.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:51 smithi155 ceph-mon[37413]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:51 smithi161 ceph-mon[37576]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:51 smithi062 ceph-mon[32993]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-16T19:18:52.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.16 MB/sec execute 311 sec latency 7574.012 ms 2024-08-16T19:18:53.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.15 MB/sec execute 312 sec latency 8574.125 ms 2024-08-16T19:18:54.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:53 smithi155 ceph-mon[37413]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 847 KiB/s wr, 99 op/s 2024-08-16T19:18:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:53 smithi161 ceph-mon[37576]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 847 KiB/s wr, 99 op/s 2024-08-16T19:18:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:53 smithi062 ceph-mon[32993]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 847 KiB/s wr, 99 op/s 2024-08-16T19:18:54.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.14 MB/sec execute 313 sec latency 9574.225 ms 2024-08-16T19:18:55.974 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.12 MB/sec execute 314 sec latency 10574.304 ms 2024-08-16T19:18:56.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:55 smithi155 ceph-mon[37413]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 KiB/s wr, 106 op/s 2024-08-16T19:18:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:55 smithi161 ceph-mon[37576]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 KiB/s wr, 106 op/s 2024-08-16T19:18:56.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:55 smithi062 ceph-mon[32993]: pgmap v260: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 KiB/s wr, 106 op/s 2024-08-16T19:18:56.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.11 MB/sec execute 315 sec latency 11574.404 ms 2024-08-16T19:18:57.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.10 MB/sec execute 316 sec latency 12574.512 ms 2024-08-16T19:18:58.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:57 smithi155 ceph-mon[37413]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:18:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:57 smithi161 ceph-mon[37576]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:18:58.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:57 smithi062 ceph-mon[32993]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:18:58.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.08 MB/sec execute 317 sec latency 13574.636 ms 2024-08-16T19:18:59.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.07 MB/sec execute 318 sec latency 14574.689 ms 2024-08-16T19:19:00.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:18:59 smithi155 ceph-mon[37413]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:19:00.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:18:59 smithi161 ceph-mon[37576]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:19:00.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:18:59 smithi062 ceph-mon[32993]: pgmap v262: 129 pgs: 129 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 204 KiB/s wr, 39 op/s 2024-08-16T19:19:00.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.06 MB/sec execute 319 sec latency 15574.816 ms 2024-08-16T19:19:01.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.05 MB/sec execute 320 sec latency 16574.931 ms 2024-08-16T19:19:02.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:01 smithi155 ceph-mon[37413]: pgmap v263: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 721 KiB/s wr, 57 op/s 2024-08-16T19:19:02.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:01 smithi161 ceph-mon[37576]: pgmap v263: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 721 KiB/s wr, 57 op/s 2024-08-16T19:19:02.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:01 smithi062 ceph-mon[32993]: pgmap v263: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 721 KiB/s wr, 57 op/s 2024-08-16T19:19:02.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.03 MB/sec execute 321 sec latency 17575.068 ms 2024-08-16T19:19:03.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.02 MB/sec execute 322 sec latency 18575.179 ms 2024-08-16T19:19:04.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:03 smithi155 ceph-mon[37413]: pgmap v264: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 533 KiB/s wr, 24 op/s 2024-08-16T19:19:04.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:03 smithi161 ceph-mon[37576]: pgmap v264: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 533 KiB/s wr, 24 op/s 2024-08-16T19:19:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:03 smithi062 ceph-mon[32993]: pgmap v264: 129 pgs: 129 active+clean; 265 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 533 KiB/s wr, 24 op/s 2024-08-16T19:19:04.975 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.01 MB/sec execute 323 sec latency 19575.298 ms 2024-08-16T19:19:05.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 4.00 MB/sec execute 324 sec latency 20575.430 ms 2024-08-16T19:19:06.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:05 smithi155 ceph-mon[37413]: pgmap v265: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 624 KiB/s wr, 54 op/s 2024-08-16T19:19:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:05 smithi155 ceph-mon[37413]: osdmap e98: 12 total, 12 up, 12 in 2024-08-16T19:19:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:05 smithi161 ceph-mon[37576]: pgmap v265: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 624 KiB/s wr, 54 op/s 2024-08-16T19:19:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:05 smithi161 ceph-mon[37576]: osdmap e98: 12 total, 12 up, 12 in 2024-08-16T19:19:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:05 smithi062 ceph-mon[32993]: pgmap v265: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 624 KiB/s wr, 54 op/s 2024-08-16T19:19:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:05 smithi062 ceph-mon[32993]: osdmap e98: 12 total, 12 up, 12 in 2024-08-16T19:19:06.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.98 MB/sec execute 325 sec latency 21575.552 ms 2024-08-16T19:19:07.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.97 MB/sec execute 326 sec latency 22575.680 ms 2024-08-16T19:19:08.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:07 smithi161 ceph-mon[37576]: pgmap v267: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:07 smithi062 ceph-mon[32993]: pgmap v267: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:08.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:07 smithi155 ceph-mon[37413]: pgmap v267: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:08.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.96 MB/sec execute 327 sec latency 23575.772 ms 2024-08-16T19:19:09.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.95 MB/sec execute 328 sec latency 24575.878 ms 2024-08-16T19:19:10.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:09 smithi161 ceph-mon[37576]: pgmap v268: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:10.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:09 smithi062 ceph-mon[32993]: pgmap v268: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:09 smithi155 ceph-mon[37413]: pgmap v268: 129 pgs: 129 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 729 KiB/s wr, 56 op/s 2024-08-16T19:19:10.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:19:10.879 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:19:10.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.93 MB/sec execute 329 sec latency 25575.969 ms 2024-08-16T19:19:11.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 -- 172.21.15.62:0/1103521387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea0101150 msgr2=0x7f2ea010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:11.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/1103521387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea0101150 0x7f2ea010d680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009930 tx=0x7f2e9802a420 comp rx=0 tx=0).stop 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 -- 172.21.15.62:0/1103521387 shutdown_connections 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/1103521387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea0101150 0x7f2ea010d680 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/1103521387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01007b0 0x7f2ea0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/1103521387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea01067b0 0x7f2ea0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 -- 172.21.15.62:0/1103521387 >> 172.21.15.62:0/1103521387 conn(0x7f2ea00fc4d0 msgr2=0x7f2ea00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 -- 172.21.15.62:0/1103521387 shutdown_connections 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.366+0000 7f2ea80ec640 1 -- 172.21.15.62:0/1103521387 wait complete. 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 Processor -- start 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 -- start start 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea0101150 0x7f2ea01028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 0x7f2ea0102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea0110c00 con 0x7f2ea01007b0 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea0110a80 con 0x7f2ea01067b0 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea80ec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea0110d80 con 0x7f2ea0101150 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea5e88640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea5e88640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58972/0 (socket says 172.21.15.62:58972) 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.367+0000 7f2ea5e88640 1 -- 172.21.15.62:0/3997726371 learned_addr learned my addr 172.21.15.62:0/3997726371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5687640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea0101150 0x7f2ea01028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea6689640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 0x7f2ea0102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 -- 172.21.15.62:0/3997726371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea0101150 msgr2=0x7f2ea01028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea0101150 0x7f2ea01028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 -- 172.21.15.62:0/3997726371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 msgr2=0x7f2ea0102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 0x7f2ea0102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 -- 172.21.15.62:0/3997726371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea01a6550 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea6689640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 0x7f2ea0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea5e88640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cce0 tx=0x7f2e9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000f650 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e9000fcd0 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea01a6840 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90018c70 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.368+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea01a6d80 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.369+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea0100c10 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.370+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2e9000f7f0 con 0x7f2ea01007b0 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.371+0000 7f2e9effd640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 0x7f2e880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:11.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.371+0000 7f2ea5687640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 0x7f2e880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:11.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.371+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9613+0+0 (secure 0 0 0) 0x7f2e900d0070 con 0x7f2ea01007b0 2024-08-16T19:19:11.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.371+0000 7f2ea5687640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 0x7f2e880a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0103e10 tx=0x7f2e9400a4c0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:11.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.374+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2e90014030 con 0x7f2ea01007b0 2024-08-16T19:19:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.588+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ea01038b0 con 0x7f2ea01007b0 2024-08-16T19:19:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.589+0000 7f2e9effd640 1 -- 172.21.15.62:0/3997726371 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11996 (secure 0 0 0) 0x7f2e9009c8e0 con 0x7f2ea01007b0 2024-08-16T19:19:11.590 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:19:11.590 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":48,"btime":"2024-08-16T19:19:10:834503+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:19:10.310082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:19:11.590 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 48 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 msgr2=0x7f2e880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 0x7f2e880a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0103e10 tx=0x7f2e9400a4c0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 msgr2=0x7f2ea0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000cce0 tx=0x7f2e9000a620 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 shutdown_connections 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f2e880a2cf0 0x7f2e880a51b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ea01067b0 0x7f2ea0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ea0101150 0x7f2ea01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 --2- 172.21.15.62:0/3997726371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea01007b0 0x7f2ea0102380 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:11.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 >> 172.21.15.62:0/3997726371 conn(0x7f2ea00fc4d0 msgr2=0x7f2ea00fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:11.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.591+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 shutdown_connections 2024-08-16T19:19:11.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:11.592+0000 7f2ea80ec640 1 -- 172.21.15.62:0/3997726371 wait complete. 2024-08-16T19:19:11.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.92 MB/sec execute 330 sec latency 26576.051 ms 2024-08-16T19:19:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:11 smithi161 ceph-mon[37576]: pgmap v269: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-08-16T19:19:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:11 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:11 smithi161 ceph-mon[37576]: osdmap e99: 12 total, 12 up, 12 in 2024-08-16T19:19:12.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:11 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3997726371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:19:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:11 smithi062 ceph-mon[32993]: pgmap v269: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-08-16T19:19:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:11 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:11 smithi062 ceph-mon[32993]: osdmap e99: 12 total, 12 up, 12 in 2024-08-16T19:19:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:11 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3997726371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:19:12.320 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id b43a572b --release --await' 2024-08-16T19:19:12.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id b43a572b --release --await 2024-08-16T19:19:12.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:11 smithi155 ceph-mon[37413]: pgmap v269: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 128 KiB/s wr, 42 op/s 2024-08-16T19:19:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:11 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:11 smithi155 ceph-mon[37413]: osdmap e99: 12 total, 12 up, 12 in 2024-08-16T19:19:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:11 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3997726371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:19:12.587 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:19:12.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 386361 3.91 MB/sec execute 331 sec latency 27576.092 ms 2024-08-16T19:19:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 -- 172.21.15.62:0/3533040836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 msgr2=0x7fa304106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 --2- 172.21.15.62:0/3533040836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa304106b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009930 tx=0x7fa2fc02a420 comp rx=0 tx=0).stop 2024-08-16T19:19:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 -- 172.21.15.62:0/3533040836 shutdown_connections 2024-08-16T19:19:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 --2- 172.21.15.62:0/3533040836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 0x7fa30410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 --2- 172.21.15.62:0/3533040836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa304100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 --2- 172.21.15.62:0/3533040836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa304106b80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 -- 172.21.15.62:0/3533040836 >> 172.21.15.62:0/3533040836 conn(0x7fa3040fc4c0 msgr2=0x7fa3040fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 -- 172.21.15.62:0/3533040836 shutdown_connections 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.057+0000 7fa3098e8640 1 -- 172.21.15.62:0/3533040836 wait complete. 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 Processor -- start 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 -- start start 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa3041981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 0x7fa304198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa304110ba0 con 0x7fa3041067a0 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa304110a20 con 0x7fa3041007a0 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3098e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa304110d20 con 0x7fa304101140 2024-08-16T19:19:13.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3037fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59000/0 (socket says 172.21.15.62:59000) 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3037fe640 1 -- 172.21.15.62:0/2510724012 learned_addr learned my addr 172.21.15.62:0/2510724012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 0x7fa304198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.058+0000 7fa302ffd640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa3041981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 -- 172.21.15.62:0/2510724012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 msgr2=0x7fa304198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 0x7fa304198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 -- 172.21.15.62:0/2510724012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 msgr2=0x7fa3041981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa3041981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 -- 172.21.15.62:0/2510724012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30419d180 con 0x7fa3041067a0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa302ffd640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa3041981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3037fe640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3088e6640 1 -- 172.21.15.62:0/2510724012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4017070 con 0x7fa3041067a0 2024-08-16T19:19:13.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3088e6640 1 -- 172.21.15.62:0/2510724012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f4003e50 con 0x7fa3041067a0 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3088e6640 1 -- 172.21.15.62:0/2510724012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40106c0 con 0x7fa3041067a0 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30419d470 con 0x7fa3041067a0 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.059+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3041a4d50 con 0x7fa3041067a0 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa3088e6640 1 -- 172.21.15.62:0/2510724012 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2f4003970 con 0x7fa3041067a0 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 Processor -- start 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 -- start start 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffc7990 0x5631dffc7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffd2b70 0x5631dffd4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631dff2dc60 con 0x5631dffc82b0 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631dfec5b90 con 0x5631dffd2b70 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa30bce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631dff29c20 con 0x5631dffc7990 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59012/0 (socket says 172.21.15.62:59012) 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 -- 172.21.15.62:0/429572327 learned_addr learned my addr 172.21.15.62:0/429572327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa3027fc640 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffd2b70 0x5631dffd4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa3027fc640 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffc7990 0x5631dffc7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa3088e6640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 0x7fa2f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 -- 172.21.15.62:0/429572327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffc7990 msgr2=0x5631dffc7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffc7990 0x5631dffc7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 -- 172.21.15.62:0/429572327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffd2b70 msgr2=0x5631dffd4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffd2b70 0x5631dffd4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.062+0000 7fa302ffd640 1 -- 172.21.15.62:0/429572327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dfc3d580 con 0x5631dffc82b0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa3037fe640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 0x7fa2f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa3088e6640 1 -- 172.21.15.62:0/2510724012 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fa2f40cf4e0 con 0x7fa3041067a0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa302ffd640 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009930 tx=0x7fa2fc032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb57574bd90ec06 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa2f9ffb640 1 -- 172.21.15.62:0/429572327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc03f070 con 0x5631dffc82b0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa3037fe640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 0x7fa2f00a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2f409b310 tx=0x7fa2f4090aa0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa2f9ffb640 1 -- 172.21.15.62:0/429572327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc0317b0 con 0x5631dffc82b0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.063+0000 7fa2f9ffb640 1 -- 172.21.15.62:0/429572327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc035070 con 0x5631dffc82b0 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- 172.21.15.62:0/429572327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 msgr2=0x5631dffd2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009930 tx=0x7fa2fc032cb0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- 172.21.15.62:0/429572327 shutdown_connections 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffd2b70 0x5631dffd4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd2630 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 --2- 172.21.15.62:0/429572327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffc7990 0x5631dffc7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- 172.21.15.62:0/429572327 >> 172.21.15.62:0/429572327 conn(0x5631dffc3f20 msgr2=0x5631dffc4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- 172.21.15.62:0/429572327 shutdown_connections 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- 172.21.15.62:0/429572327 wait complete. 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 -- start start 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.064+0000 7fa30bce6740 1 Processor -- start 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffcfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631dffd5740 con 0x5631dffc82b0 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5631dfecb7c0 con 0x5631dffc7990 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5631dffd58c0 con 0x5631dffd2b70 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa30bce6740 10 client.0 _ll_register_callbacks cb 0x5631dfed4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:19:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59020/0 (socket says 172.21.15.62:59020) 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 -- 172.21.15.62:0/2608817551 learned_addr learned my addr 172.21.15.62:0/2608817551 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffcfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 -- 172.21.15.62:0/2608817551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 msgr2=0x5631dffd0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 -- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 msgr2=0x5631dffcfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffcfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 -- 172.21.15.62:0/2608817551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dfd07960 con 0x5631dffc82b0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffcfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa302ffd640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc02a940 tx=0x7fa2fc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3322bc278ff48c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.065+0000 7fa2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:19:13.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc035070 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2fc0317b0 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fa2fc0048d0 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc055050 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 10 client.15948 fetch_fsmap learned FSMap version 48 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5631dffe17b0 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc0317b0 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa2fc03bc80 con 0x5631dffc82b0 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 10 client.15948 fetch_fsmap finished waiting for FSMap version 48 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 10 client.15948 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 20 client.15948 populate_metadata read hostname 'smithi062' 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631dffe2b00 0x5631dffe4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 4 client.15948 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:19:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631dff3b1d0 con 0x5631dffe2b00 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.066+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631dffe2b00 0x5631dffe4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.067+0000 7fa3027fc640 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631dffe2b00 0x5631dffe4ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de2ddf76329fb025 server_cookie=4f27248c6905cde7 in_seq=0 out_seq=0 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.067+0000 7fa2e2ffd640 10 client.15948 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/2608817551 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631dff3b1d0 con 0x5631dffe2b00 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa2e2ffd640 10 client.15948 handle_command_reply: tid=0 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 client.15948 shutdown 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 msgr2=0x5631dffcffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc02a940 tx=0x7fa2fc004610 comp rx=0 tx=0).stop 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 shutdown_connections 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631dffe2b00 0x5631dffe4ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffcffc0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 --2- 172.21.15.62:0/2608817551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffcfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa2e2ffd640 0 client.15948 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 >> 172.21.15.62:0/2608817551 conn(0x5631dffc3f20 msgr2=0x5631dffc9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 shutdown_connections 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 1 -- 172.21.15.62:0/2608817551 wait complete. 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.068+0000 7fa30bce6740 20 client.15948 trim_cache size 0 max 16384 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 Processor -- start 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 -- start start 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffc2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffeab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041aee00 con 0x5631dffc82b0 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b0a00 con 0x5631dffc7990 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa30bce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041af4d0 con 0x5631dffd2b70 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59022/0 (socket says 172.21.15.62:59022) 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.085+0000 7fa302ffd640 1 -- 172.21.15.62:0/4090089981 learned_addr learned my addr 172.21.15.62:0/4090089981 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa3027fc640 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffc2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 -- 172.21.15.62:0/4090089981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 msgr2=0x5631dffeab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffeab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 -- 172.21.15.62:0/4090089981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 msgr2=0x5631dffc2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffc2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 -- 172.21.15.62:0/4090089981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dffc2470 con 0x5631dffc82b0 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa3027fc640 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffc2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa302ffd640 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc033000 tx=0x7fa2fc005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1399362f13fc64 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa2f97fa640 1 -- 172.21.15.62:0/4090089981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc05e070 con 0x5631dffc82b0 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa2f97fa640 1 -- 172.21.15.62:0/4090089981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc04c070 con 0x5631dffc82b0 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.086+0000 7fa2f97fa640 1 -- 172.21.15.62:0/4090089981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc035040 con 0x5631dffc82b0 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- 172.21.15.62:0/4090089981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 msgr2=0x5631dffea5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc033000 tx=0x7fa2fc005410 comp rx=0 tx=0).stop 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- 172.21.15.62:0/4090089981 shutdown_connections 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffeab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffea5e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- 172.21.15.62:0/4090089981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffc2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- 172.21.15.62:0/4090089981 >> 172.21.15.62:0/4090089981 conn(0x5631dffc3f20 msgr2=0x5631dffc2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- 172.21.15.62:0/4090089981 shutdown_connections 2024-08-16T19:19:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- 172.21.15.62:0/4090089981 wait complete. 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- start start 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 Processor -- start 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffdb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041b0270 con 0x5631dffc82b0 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041afba0 con 0x5631dffc7990 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041ace00 con 0x5631dffd2b70 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.087+0000 7fa30bce6740 10 client.0 _ll_register_callbacks cb 0x7fa2fc031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59038/0 (socket says 172.21.15.62:59038) 2024-08-16T19:19:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 -- 172.21.15.62:0/1263845220 learned_addr learned my addr 172.21.15.62:0/1263845220 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:19:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa3027fc640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffdb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa3027fc640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 -- 172.21.15.62:0/1263845220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 msgr2=0x5631dffd7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 -- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 msgr2=0x5631dffdb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffdb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 -- 172.21.15.62:0/1263845220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dffd8290 con 0x5631dffc82b0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa3027fc640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa302ffd640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90f05060def95d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa2e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc05e040 con 0x5631dffc82b0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fa2fc004030 con 0x5631dffc82b0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2fc04c070 con 0x5631dffc82b0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa2fc06a050 con 0x5631dffc82b0 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa30bce6740 10 client.15954 fetch_fsmap learned FSMap version 48 2024-08-16T19:19:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.088+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5631e004e500 con 0x5631dffc82b0 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2fc04c070 con 0x5631dffc82b0 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa2fc004b80 con 0x5631dffc82b0 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 10 client.15954 fetch_fsmap finished waiting for FSMap version 48 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 10 client.15954 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 20 client.15954 populate_metadata read hostname 'smithi062' 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631e004eba0 0x5631e0050fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 4 client.15954 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b43a572b", "release": true, "await": true}] 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b43a572b", "release": true, "await": true}) -- 0x5631dff356e0 con 0x5631e004eba0 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.089+0000 7fa3027fc640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631e004eba0 0x5631e0050fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.090+0000 7fa2e2ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:19:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.090+0000 7fa3027fc640 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631e004eba0 0x5631e0050fc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75885d66c1387594 server_cookie=b26a2b0966b91172 in_seq=0 out_seq=0 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 48, 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 36, 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "b43a572b": { 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 36, 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:19:13.129 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.7, 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:19:13.130 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa2e2ffd640 1 -- 172.21.15.62:0/1263845220 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5631dff356e0 con 0x5631e004eba0 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa2e2ffd640 10 client.15954 handle_command_reply: tid=0 2024-08-16T19:19:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 client.15954 shutdown 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 msgr2=0x5631dffd7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc006010 tx=0x7fa2fc0317e0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 shutdown_connections 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5631e004eba0 0x5631e0050fc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5631dffd2b70 0x5631dffd7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631dffc82b0 0x5631dffd7520 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa2e2ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 --2- 172.21.15.62:0/1263845220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5631dffc7990 0x5631dffdb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 >> 172.21.15.62:0/1263845220 conn(0x5631dffc3f20 msgr2=0x5631dffc3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 shutdown_connections 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 1 -- 172.21.15.62:0/1263845220 wait complete. 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.102+0000 7fa30bce6740 20 client.15954 trim_cache size 0 max 16384 2024-08-16T19:19:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 msgr2=0x7fa2f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 0x7fa2f00a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2f409b310 tx=0x7fa2f4090aa0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 msgr2=0x7fa30419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009fd0 tx=0x7fa2f400b260 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 shutdown_connections 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa2f00a2d70 0x7fa2f00a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3041067a0 0x7fa30419caa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa304101140 0x7fa304198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 --2- 172.21.15.62:0/2510724012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3041007a0 0x7fa3041981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 >> 172.21.15.62:0/2510724012 conn(0x7fa3040fc4c0 msgr2=0x7fa30406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:19:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 shutdown_connections 2024-08-16T19:19:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:19:13.103+0000 7fa3098e8640 1 -- 172.21.15.62:0/2510724012 wait complete. 2024-08-16T19:19:13.830 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b43a572b', total seconds elapsed: 31.0 2024-08-16T19:19:13.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 387482 3.92 MB/sec execute 332 sec latency 27692.981 ms 2024-08-16T19:19:14.130 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:13 smithi155 ceph-mon[37413]: pgmap v271: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-08-16T19:19:14.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:13 smithi161 ceph-mon[37576]: pgmap v271: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-08-16T19:19:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:13 smithi062 ceph-mon[32993]: pgmap v271: 129 pgs: 129 active+clean; 261 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-08-16T19:19:14.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 389685 3.94 MB/sec execute 333 sec latency 12.791 ms 2024-08-16T19:19:15.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 391010 3.95 MB/sec execute 334 sec latency 46.459 ms 2024-08-16T19:19:16.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:15 smithi161 ceph-mon[37576]: pgmap v272: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 298 KiB/s wr, 45 op/s 2024-08-16T19:19:16.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:15 smithi062 ceph-mon[32993]: pgmap v272: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 298 KiB/s wr, 45 op/s 2024-08-16T19:19:16.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:15 smithi155 ceph-mon[37413]: pgmap v272: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 298 KiB/s wr, 45 op/s 2024-08-16T19:19:16.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 393200 3.98 MB/sec execute 335 sec latency 20.123 ms 2024-08-16T19:19:17.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 394576 3.99 MB/sec execute 336 sec latency 14.384 ms 2024-08-16T19:19:18.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:17 smithi161 ceph-mon[37576]: pgmap v273: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:18.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:17 smithi062 ceph-mon[32993]: pgmap v273: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:18.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:17 smithi155 ceph-mon[37413]: pgmap v273: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:18.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 396375 4.00 MB/sec execute 337 sec latency 14.942 ms 2024-08-16T19:19:19.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 397902 4.02 MB/sec execute 338 sec latency 12.073 ms 2024-08-16T19:19:20.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:19 smithi161 ceph-mon[37576]: pgmap v274: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:20.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:19 smithi062 ceph-mon[32993]: pgmap v274: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:20.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:19 smithi155 ceph-mon[37413]: pgmap v274: 129 pgs: 129 active+clean; 263 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 287 KiB/s wr, 43 op/s 2024-08-16T19:19:20.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 399678 4.03 MB/sec execute 339 sec latency 15.750 ms 2024-08-16T19:19:21.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 401273 4.05 MB/sec execute 340 sec latency 18.088 ms 2024-08-16T19:19:22.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:21 smithi161 ceph-mon[37576]: pgmap v275: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-08-16T19:19:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:21 smithi062 ceph-mon[32993]: pgmap v275: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-08-16T19:19:22.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:21 smithi155 ceph-mon[37413]: pgmap v275: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-08-16T19:19:22.976 INFO:tasks.workunit.client.0.smithi062.stdout: 1 403194 4.06 MB/sec execute 341 sec latency 16.075 ms 2024-08-16T19:19:23.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 404827 4.08 MB/sec execute 342 sec latency 21.005 ms 2024-08-16T19:19:24.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:23 smithi161 ceph-mon[37576]: pgmap v276: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-08-16T19:19:24.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:23 smithi062 ceph-mon[32993]: pgmap v276: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-08-16T19:19:24.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:23 smithi155 ceph-mon[37413]: pgmap v276: 129 pgs: 129 active+clean; 273 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-08-16T19:19:24.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 406274 4.09 MB/sec execute 343 sec latency 14.142 ms 2024-08-16T19:19:25.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 408333 4.11 MB/sec execute 344 sec latency 22.724 ms 2024-08-16T19:19:26.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:25 smithi161 ceph-mon[37576]: pgmap v277: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-08-16T19:19:26.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:25 smithi062 ceph-mon[32993]: pgmap v277: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-08-16T19:19:26.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:25 smithi155 ceph-mon[37413]: pgmap v277: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-08-16T19:19:26.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 409641 4.12 MB/sec execute 345 sec latency 45.064 ms 2024-08-16T19:19:27.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 411857 4.14 MB/sec execute 346 sec latency 38.675 ms 2024-08-16T19:19:28.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:27 smithi161 ceph-mon[37576]: pgmap v278: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:28.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:27 smithi062 ceph-mon[32993]: pgmap v278: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:28.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:27 smithi155 ceph-mon[37413]: pgmap v278: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:28.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 413530 4.15 MB/sec execute 347 sec latency 15.787 ms 2024-08-16T19:19:29.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 415463 4.18 MB/sec execute 348 sec latency 14.372 ms 2024-08-16T19:19:30.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:29 smithi161 ceph-mon[37576]: pgmap v279: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:29 smithi062 ceph-mon[32993]: pgmap v279: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:30.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:29 smithi155 ceph-mon[37413]: pgmap v279: 129 pgs: 129 active+clean; 281 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-08-16T19:19:30.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 416866 4.18 MB/sec execute 349 sec latency 14.822 ms 2024-08-16T19:19:31.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 418927 4.21 MB/sec execute 350 sec latency 50.104 ms 2024-08-16T19:19:32.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:31 smithi161 ceph-mon[37576]: pgmap v280: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-16T19:19:32.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:31 smithi062 ceph-mon[32993]: pgmap v280: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-16T19:19:32.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:31 smithi155 ceph-mon[37413]: pgmap v280: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-08-16T19:19:32.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 420162 4.21 MB/sec execute 351 sec latency 25.012 ms 2024-08-16T19:19:33.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 422376 4.24 MB/sec execute 352 sec latency 18.924 ms 2024-08-16T19:19:34.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:33 smithi161 ceph-mon[37576]: pgmap v281: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-08-16T19:19:34.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:33 smithi062 ceph-mon[32993]: pgmap v281: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-08-16T19:19:34.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:33 smithi155 ceph-mon[37413]: pgmap v281: 129 pgs: 129 active+clean; 291 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-08-16T19:19:34.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 423651 4.24 MB/sec execute 353 sec latency 13.346 ms 2024-08-16T19:19:35.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 425909 4.27 MB/sec execute 354 sec latency 13.220 ms 2024-08-16T19:19:36.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:35 smithi161 ceph-mon[37576]: pgmap v282: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-08-16T19:19:36.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:35 smithi062 ceph-mon[32993]: pgmap v282: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-08-16T19:19:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:35 smithi155 ceph-mon[37413]: pgmap v282: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 318 op/s 2024-08-16T19:19:36.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 427166 4.27 MB/sec execute 355 sec latency 17.300 ms 2024-08-16T19:19:37.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 429341 4.29 MB/sec execute 356 sec latency 13.451 ms 2024-08-16T19:19:38.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:37 smithi161 ceph-mon[37576]: pgmap v283: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:37 smithi062 ceph-mon[32993]: pgmap v283: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:38.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:37 smithi155 ceph-mon[37413]: pgmap v283: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:38.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 430429 4.29 MB/sec execute 357 sec latency 21.333 ms 2024-08-16T19:19:39.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:19:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:19:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:19:39.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:19:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:19:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:19:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:19:39.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:19:39.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:19:39.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:19:39.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:19:39.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:19:39.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 432763 4.32 MB/sec execute 358 sec latency 14.433 ms 2024-08-16T19:19:40.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:39 smithi161 ceph-mon[37576]: pgmap v284: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:40.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:39 smithi062 ceph-mon[32993]: pgmap v284: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:40.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:39 smithi155 ceph-mon[37413]: pgmap v284: 129 pgs: 129 active+clean; 300 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-08-16T19:19:40.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 433929 4.32 MB/sec execute 359 sec latency 19.100 ms 2024-08-16T19:19:41.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:41 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:41.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:41 smithi161 ceph-mon[37576]: pgmap v285: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-08-16T19:19:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:41 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:41.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:41 smithi062 ceph-mon[32993]: pgmap v285: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-08-16T19:19:41.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:41 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:41.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:41 smithi155 ceph-mon[37413]: pgmap v285: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-08-16T19:19:41.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 435971 4.34 MB/sec execute 360 sec latency 16.009 ms 2024-08-16T19:19:42.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 437351 4.35 MB/sec execute 361 sec latency 12.420 ms 2024-08-16T19:19:43.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:43 smithi155 ceph-mon[37413]: pgmap v286: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-08-16T19:19:43.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:43 smithi161 ceph-mon[37576]: pgmap v286: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-08-16T19:19:43.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 439284 4.37 MB/sec execute 362 sec latency 28.223 ms 2024-08-16T19:19:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:43 smithi062 ceph-mon[32993]: pgmap v286: 129 pgs: 129 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-08-16T19:19:44.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 440646 4.37 MB/sec execute 363 sec latency 18.641 ms 2024-08-16T19:19:45.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:45 smithi155 ceph-mon[37413]: pgmap v287: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-08-16T19:19:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:45 smithi161 ceph-mon[37576]: pgmap v287: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-08-16T19:19:45.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 442877 4.39 MB/sec execute 364 sec latency 15.086 ms 2024-08-16T19:19:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:45 smithi062 ceph-mon[32993]: pgmap v287: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.6 MiB/s wr, 314 op/s 2024-08-16T19:19:46.977 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.40 MB/sec execute 365 sec latency 190.743 ms 2024-08-16T19:19:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:47 smithi155 ceph-mon[37413]: pgmap v288: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:47.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:47 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:47.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:47 smithi161 ceph-mon[37576]: pgmap v288: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:47.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:47 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:47.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.39 MB/sec execute 366 sec latency 1190.869 ms 2024-08-16T19:19:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:47 smithi062 ceph-mon[32993]: pgmap v288: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:47 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:48.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.38 MB/sec execute 367 sec latency 2190.961 ms 2024-08-16T19:19:49.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:49 smithi155 ceph-mon[37413]: pgmap v289: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:49 smithi161 ceph-mon[37576]: pgmap v289: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:49.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.36 MB/sec execute 368 sec latency 3191.055 ms 2024-08-16T19:19:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:49 smithi062 ceph-mon[32993]: pgmap v289: 129 pgs: 129 active+clean; 318 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-16T19:19:50.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.35 MB/sec execute 369 sec latency 4191.190 ms 2024-08-16T19:19:51.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:51 smithi155 ceph-mon[37413]: pgmap v290: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-08-16T19:19:51.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:51 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:51.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:51 smithi161 ceph-mon[37576]: pgmap v290: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-08-16T19:19:51.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:51 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:51.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.34 MB/sec execute 370 sec latency 5191.302 ms 2024-08-16T19:19:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:51 smithi062 ceph-mon[32993]: pgmap v290: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-08-16T19:19:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:51 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:19:52.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.33 MB/sec execute 371 sec latency 6191.397 ms 2024-08-16T19:19:53.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:53 smithi155 ceph-mon[37413]: pgmap v291: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-08-16T19:19:53.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:53 smithi161 ceph-mon[37576]: pgmap v291: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-08-16T19:19:53.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.32 MB/sec execute 372 sec latency 7191.464 ms 2024-08-16T19:19:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:53 smithi062 ceph-mon[32993]: pgmap v291: 129 pgs: 129 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-08-16T19:19:54.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.31 MB/sec execute 373 sec latency 8191.564 ms 2024-08-16T19:19:55.799 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:55 smithi062 ceph-mon[32993]: pgmap v292: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-08-16T19:19:55.831 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:55 smithi155 ceph-mon[37413]: pgmap v292: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-08-16T19:19:55.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:55 smithi161 ceph-mon[37576]: pgmap v292: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-08-16T19:19:55.978 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.29 MB/sec execute 374 sec latency 9191.692 ms 2024-08-16T19:19:56.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.28 MB/sec execute 375 sec latency 10191.760 ms 2024-08-16T19:19:57.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:57 smithi155 ceph-mon[37413]: pgmap v293: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:19:57.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:57 smithi161 ceph-mon[37576]: pgmap v293: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:19:57.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.27 MB/sec execute 376 sec latency 11191.811 ms 2024-08-16T19:19:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:57 smithi062 ceph-mon[32993]: pgmap v293: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:19:58.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.26 MB/sec execute 377 sec latency 12192.188 ms 2024-08-16T19:19:59.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:19:59 smithi155 ceph-mon[37413]: pgmap v294: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:19:59.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:19:59 smithi161 ceph-mon[37576]: pgmap v294: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:19:59.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444017 4.25 MB/sec execute 378 sec latency 13192.324 ms 2024-08-16T19:19:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:19:59 smithi062 ceph-mon[32993]: pgmap v294: 129 pgs: 129 active+clean; 329 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-16T19:20:00.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:00 smithi155 ceph-mon[37413]: overall HEALTH_OK 2024-08-16T19:20:00.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:00 smithi161 ceph-mon[37576]: overall HEALTH_OK 2024-08-16T19:20:00.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444142 4.24 MB/sec execute 379 sec latency 14096.999 ms 2024-08-16T19:20:00.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:00 smithi062 ceph-mon[32993]: overall HEALTH_OK 2024-08-16T19:20:01.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:01 smithi155 ceph-mon[37413]: pgmap v295: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-08-16T19:20:01.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:01 smithi161 ceph-mon[37576]: pgmap v295: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-08-16T19:20:01.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 444814 4.24 MB/sec execute 380 sec latency 71.310 ms 2024-08-16T19:20:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:01 smithi062 ceph-mon[32993]: pgmap v295: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-08-16T19:20:02.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 447100 4.26 MB/sec execute 381 sec latency 16.297 ms 2024-08-16T19:20:03.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:03 smithi155 ceph-mon[37413]: pgmap v296: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:20:03.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:03 smithi161 ceph-mon[37576]: pgmap v296: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:20:03.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 448371 4.27 MB/sec execute 382 sec latency 13.106 ms 2024-08-16T19:20:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:03 smithi062 ceph-mon[32993]: pgmap v296: 129 pgs: 129 active+clean; 326 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-16T19:20:04.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 450153 4.28 MB/sec execute 383 sec latency 33.599 ms 2024-08-16T19:20:05.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:05 smithi155 ceph-mon[37413]: pgmap v297: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-08-16T19:20:05.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:05 smithi155 ceph-mon[37413]: osdmap e100: 12 total, 12 up, 12 in 2024-08-16T19:20:05.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:05 smithi161 ceph-mon[37576]: pgmap v297: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-08-16T19:20:05.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:05 smithi161 ceph-mon[37576]: osdmap e100: 12 total, 12 up, 12 in 2024-08-16T19:20:05.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 451398 4.28 MB/sec execute 384 sec latency 21.932 ms 2024-08-16T19:20:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:05 smithi062 ceph-mon[32993]: pgmap v297: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 176 op/s 2024-08-16T19:20:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:05 smithi062 ceph-mon[32993]: osdmap e100: 12 total, 12 up, 12 in 2024-08-16T19:20:06.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 453272 4.30 MB/sec execute 385 sec latency 15.400 ms 2024-08-16T19:20:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:07 smithi155 ceph-mon[37413]: pgmap v299: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:07.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:07 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:07 smithi161 ceph-mon[37576]: pgmap v299: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:07 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:07.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 454657 4.31 MB/sec execute 386 sec latency 15.185 ms 2024-08-16T19:20:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:07 smithi062 ceph-mon[32993]: pgmap v299: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:07 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:08.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 456494 4.32 MB/sec execute 387 sec latency 13.691 ms 2024-08-16T19:20:09.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:09 smithi155 ceph-mon[37413]: pgmap v300: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:09 smithi161 ceph-mon[37576]: pgmap v300: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:09.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458025 4.34 MB/sec execute 388 sec latency 47.612 ms 2024-08-16T19:20:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:09 smithi062 ceph-mon[32993]: pgmap v300: 129 pgs: 129 active+clean; 333 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-16T19:20:10.979 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458757 4.36 MB/sec execute 389 sec latency 63.406 ms 2024-08-16T19:20:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:11 smithi161 ceph-mon[37576]: pgmap v301: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-08-16T19:20:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:11 smithi161 ceph-mon[37576]: osdmap e101: 12 total, 12 up, 12 in 2024-08-16T19:20:11.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458757 4.35 MB/sec execute 390 sec latency 1063.527 ms 2024-08-16T19:20:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:11 smithi062 ceph-mon[32993]: pgmap v301: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-08-16T19:20:11.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:11 smithi062 ceph-mon[32993]: osdmap e101: 12 total, 12 up, 12 in 2024-08-16T19:20:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:11 smithi155 ceph-mon[37413]: pgmap v301: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 281 op/s 2024-08-16T19:20:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:11 smithi155 ceph-mon[37413]: osdmap e101: 12 total, 12 up, 12 in 2024-08-16T19:20:12.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458757 4.34 MB/sec execute 391 sec latency 2063.654 ms 2024-08-16T19:20:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:13 smithi161 ceph-mon[37576]: pgmap v303: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-08-16T19:20:13.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458757 4.33 MB/sec execute 392 sec latency 3063.767 ms 2024-08-16T19:20:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:13 smithi062 ceph-mon[32993]: pgmap v303: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-08-16T19:20:14.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:13 smithi155 ceph-mon[37413]: pgmap v303: 129 pgs: 129 active+clean; 337 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-08-16T19:20:14.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:14 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:14.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458757 4.32 MB/sec execute 393 sec latency 4063.863 ms 2024-08-16T19:20:14.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:14 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:15.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:14 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:15.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:15 smithi161 ceph-mon[37576]: pgmap v304: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-08-16T19:20:15.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.31 MB/sec execute 394 sec latency 4952.131 ms 2024-08-16T19:20:15.984 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:15 smithi155 ceph-mon[37413]: pgmap v304: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-08-16T19:20:16.012 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:15 smithi062 ceph-mon[32993]: pgmap v304: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 209 op/s 2024-08-16T19:20:16.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.30 MB/sec execute 395 sec latency 1073.348 ms 2024-08-16T19:20:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:17 smithi161 ceph-mon[37576]: pgmap v305: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:17.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.29 MB/sec execute 396 sec latency 2073.474 ms 2024-08-16T19:20:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:17 smithi062 ceph-mon[32993]: pgmap v305: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:17 smithi155 ceph-mon[37413]: pgmap v305: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:18.980 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.28 MB/sec execute 397 sec latency 3073.582 ms 2024-08-16T19:20:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:19 smithi161 ceph-mon[37576]: pgmap v306: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:19.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.27 MB/sec execute 398 sec latency 4073.687 ms 2024-08-16T19:20:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:19 smithi062 ceph-mon[32993]: pgmap v306: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:19 smithi155 ceph-mon[37413]: pgmap v306: 129 pgs: 129 active+clean; 334 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-08-16T19:20:20.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.26 MB/sec execute 399 sec latency 5073.797 ms 2024-08-16T19:20:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:21 smithi161 ceph-mon[37576]: pgmap v307: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-16T19:20:21.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.25 MB/sec execute 400 sec latency 6073.925 ms 2024-08-16T19:20:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:21 smithi062 ceph-mon[32993]: pgmap v307: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-16T19:20:22.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:21 smithi155 ceph-mon[37413]: pgmap v307: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-16T19:20:22.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 458789 4.24 MB/sec execute 401 sec latency 7074.031 ms 2024-08-16T19:20:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:23 smithi161 ceph-mon[37576]: pgmap v308: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-16T19:20:23.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 460199 4.25 MB/sec execute 402 sec latency 7330.361 ms 2024-08-16T19:20:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:23 smithi062 ceph-mon[32993]: pgmap v308: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-16T19:20:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:23 smithi155 ceph-mon[37413]: pgmap v308: 129 pgs: 129 active+clean; 333 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 1.2 MiB/s wr, 96 op/s 2024-08-16T19:20:24.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 462288 4.28 MB/sec execute 403 sec latency 10.785 ms 2024-08-16T19:20:25.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:25 smithi161 ceph-mon[37576]: pgmap v309: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-16T19:20:25.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 464263 4.28 MB/sec execute 404 sec latency 13.183 ms 2024-08-16T19:20:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:25 smithi062 ceph-mon[32993]: pgmap v309: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-16T19:20:26.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:25 smithi155 ceph-mon[37413]: pgmap v309: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-16T19:20:26.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 465992 4.30 MB/sec execute 405 sec latency 10.129 ms 2024-08-16T19:20:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:27 smithi161 ceph-mon[37576]: pgmap v310: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:27.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 468164 4.32 MB/sec execute 406 sec latency 17.732 ms 2024-08-16T19:20:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:27 smithi062 ceph-mon[32993]: pgmap v310: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:28.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:27 smithi155 ceph-mon[37413]: pgmap v310: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:28.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 469676 4.33 MB/sec execute 407 sec latency 14.988 ms 2024-08-16T19:20:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:29 smithi161 ceph-mon[37576]: pgmap v311: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:29 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:29.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 471759 4.34 MB/sec execute 408 sec latency 14.501 ms 2024-08-16T19:20:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:29 smithi062 ceph-mon[32993]: pgmap v311: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:29 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:30.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:29 smithi155 ceph-mon[37413]: pgmap v311: 129 pgs: 129 active+clean; 336 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 75 op/s 2024-08-16T19:20:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:29 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:30.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 473216 4.35 MB/sec execute 409 sec latency 16.678 ms 2024-08-16T19:20:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:31 smithi161 ceph-mon[37576]: pgmap v312: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-08-16T19:20:31.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 475434 4.37 MB/sec execute 410 sec latency 15.229 ms 2024-08-16T19:20:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:31 smithi062 ceph-mon[32993]: pgmap v312: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-08-16T19:20:32.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:31 smithi155 ceph-mon[37413]: pgmap v312: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-08-16T19:20:32.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 476825 4.38 MB/sec execute 411 sec latency 89.670 ms 2024-08-16T19:20:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:33 smithi161 ceph-mon[37576]: pgmap v313: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-16T19:20:33.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 479301 4.40 MB/sec execute 412 sec latency 12.860 ms 2024-08-16T19:20:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:33 smithi062 ceph-mon[32993]: pgmap v313: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-16T19:20:34.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:33 smithi155 ceph-mon[37413]: pgmap v313: 129 pgs: 129 active+clean; 346 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-08-16T19:20:34.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 480542 4.40 MB/sec execute 413 sec latency 26.053 ms 2024-08-16T19:20:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:35 smithi161 ceph-mon[37576]: pgmap v314: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-08-16T19:20:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:35 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:35.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 482657 4.42 MB/sec execute 414 sec latency 40.615 ms 2024-08-16T19:20:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:35 smithi062 ceph-mon[32993]: pgmap v314: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-08-16T19:20:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:35 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:36.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:35 smithi155 ceph-mon[37413]: pgmap v314: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-08-16T19:20:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:35 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:36.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 484192 4.43 MB/sec execute 415 sec latency 14.589 ms 2024-08-16T19:20:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:37 smithi161 ceph-mon[37576]: pgmap v315: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:37.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 486743 4.46 MB/sec execute 416 sec latency 15.764 ms 2024-08-16T19:20:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:37 smithi062 ceph-mon[32993]: pgmap v315: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:37 smithi155 ceph-mon[37413]: pgmap v315: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:38.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:38 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:20:38.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 487992 4.46 MB/sec execute 417 sec latency 12.294 ms 2024-08-16T19:20:38.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:38 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:20:39.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:38 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:20:39.981 INFO:tasks.workunit.client.0.smithi062.stdout: 1 489772 4.47 MB/sec execute 418 sec latency 114.781 ms 2024-08-16T19:20:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:39 smithi062 ceph-mon[32993]: pgmap v316: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:39 smithi155 ceph-mon[37413]: pgmap v316: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:40.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:39 smithi161 ceph-mon[37576]: pgmap v316: 129 pgs: 129 active+clean; 356 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-16T19:20:40.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 490928 4.48 MB/sec execute 419 sec latency 71.660 ms 2024-08-16T19:20:40.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:40.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:40.998 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:40 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:40 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.139 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:40 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:41.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 492200 4.48 MB/sec execute 420 sec latency 166.992 ms 2024-08-16T19:20:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:41 smithi062 ceph-mon[32993]: pgmap v317: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-08-16T19:20:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:41 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:20:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:41 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:20:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:41 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:20:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:41 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:42.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:41 smithi155 ceph-mon[37413]: pgmap v317: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-08-16T19:20:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:41 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:20:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:41 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:20:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:41 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:20:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:41 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:42.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:41 smithi161 ceph-mon[37576]: pgmap v317: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 309 op/s 2024-08-16T19:20:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:41 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi062 to 176.0M 2024-08-16T19:20:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:41 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi062 to 184603955: error parsing value: Value '184603955' is below minimum 939524096 2024-08-16T19:20:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:41 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi161 to 176.2M 2024-08-16T19:20:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:41 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi161 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:42.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 494210 4.50 MB/sec execute 421 sec latency 64.363 ms 2024-08-16T19:20:43.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 495767 4.50 MB/sec execute 422 sec latency 16.933 ms 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: pgmap v318: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:20:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:20:44.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: pgmap v318: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:20:44.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: pgmap v318: 129 pgs: 129 active+clean; 365 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 209 op/s 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:20:44.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:20:44.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 497781 4.53 MB/sec execute 423 sec latency 15.855 ms 2024-08-16T19:20:45.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:45 smithi161 ceph-mon[37576]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:20:45.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:45 smithi161 ceph-mon[37576]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:45.479 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:45 smithi155 ceph-mon[37413]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:20:45.480 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:45 smithi155 ceph-mon[37413]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:45 smithi062 ceph-mon[32993]: Adjusting osd_memory_target on smithi155 to 176.2M 2024-08-16T19:20:45.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:45 smithi062 ceph-mon[32993]: Unable to set osd_memory_target on smithi155 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-08-16T19:20:45.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 499227 4.53 MB/sec execute 424 sec latency 17.413 ms 2024-08-16T19:20:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:46 smithi161 ceph-mon[37576]: pgmap v319: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2024-08-16T19:20:46.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:46 smithi062 ceph-mon[32993]: pgmap v319: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2024-08-16T19:20:46.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:46 smithi155 ceph-mon[37413]: pgmap v319: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 318 op/s 2024-08-16T19:20:46.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 501306 4.55 MB/sec execute 425 sec latency 14.090 ms 2024-08-16T19:20:47.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 502802 4.55 MB/sec execute 426 sec latency 15.029 ms 2024-08-16T19:20:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:48 smithi062 ceph-mon[32993]: pgmap v320: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:48.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:48 smithi155 ceph-mon[37413]: pgmap v320: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:48.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:48 smithi161 ceph-mon[37576]: pgmap v320: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:48.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 504855 4.57 MB/sec execute 427 sec latency 40.357 ms 2024-08-16T19:20:49.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 506554 4.57 MB/sec execute 428 sec latency 19.605 ms 2024-08-16T19:20:50.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:50 smithi062 ceph-mon[32993]: pgmap v321: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:50.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:50 smithi155 ceph-mon[37413]: pgmap v321: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:50.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:50 smithi161 ceph-mon[37576]: pgmap v321: 129 pgs: 129 active+clean; 373 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-08-16T19:20:50.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 508450 4.60 MB/sec execute 429 sec latency 10.232 ms 2024-08-16T19:20:51.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 510176 4.60 MB/sec execute 430 sec latency 20.371 ms 2024-08-16T19:20:52.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:52 smithi062 ceph-mon[32993]: pgmap v322: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-08-16T19:20:52.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:52 smithi155 ceph-mon[37413]: pgmap v322: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-08-16T19:20:52.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:52 smithi161 ceph-mon[37576]: pgmap v322: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-08-16T19:20:52.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 512118 4.62 MB/sec execute 431 sec latency 10.321 ms 2024-08-16T19:20:53.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 514158 4.63 MB/sec execute 432 sec latency 17.029 ms 2024-08-16T19:20:54.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:54 smithi062 ceph-mon[32993]: pgmap v323: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-08-16T19:20:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:54 smithi155 ceph-mon[37413]: pgmap v323: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-08-16T19:20:54.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:54 smithi161 ceph-mon[37576]: pgmap v323: 129 pgs: 129 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 208 op/s 2024-08-16T19:20:54.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 515866 4.64 MB/sec execute 433 sec latency 12.671 ms 2024-08-16T19:20:55.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 517885 4.66 MB/sec execute 434 sec latency 33.320 ms 2024-08-16T19:20:56.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:56 smithi161 ceph-mon[37576]: pgmap v324: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-08-16T19:20:56.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:56 smithi062 ceph-mon[32993]: pgmap v324: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-08-16T19:20:56.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:56 smithi155 ceph-mon[37413]: pgmap v324: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 311 op/s 2024-08-16T19:20:56.716 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:20:56.717 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.95 2024-08-16T19:20:56.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:20:56.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 519266 4.66 MB/sec execute 435 sec latency 17.278 ms 2024-08-16T19:20:57.001 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:20:57.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 -- 172.21.15.62:0/1414246770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984102c20 msgr2=0x7f798410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 --2- 172.21.15.62:0/1414246770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984102c20 0x7f798410d770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f797c009ca0 tx=0x7f797c02a950 comp rx=0 tx=0).stop 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 -- 172.21.15.62:0/1414246770 shutdown_connections 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 --2- 172.21.15.62:0/1414246770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984102c20 0x7f798410d770 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 --2- 172.21.15.62:0/1414246770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79841022a0 0x7f79841026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 --2- 172.21.15.62:0/1414246770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7984100b50 0x7f7984100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 -- 172.21.15.62:0/1414246770 >> 172.21.15.62:0/1414246770 conn(0x7f79840fc500 msgr2=0x7f79840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 -- 172.21.15.62:0/1414246770 shutdown_connections 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.491+0000 7f798a66c640 1 -- 172.21.15.62:0/1414246770 wait complete. 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 Processor -- start 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 -- start start 2024-08-16T19:20:57.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 0x7f798419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7984102c20 0x7f798419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110cb0 con 0x7f7984100b50 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110b30 con 0x7f79841022a0 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f798a66c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110e30 con 0x7f7984102c20 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f7983fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f79837fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 0x7f798419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f7983fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49108/0 (socket says 172.21.15.62:49108) 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f7983fff640 1 -- 172.21.15.62:0/3449822980 learned_addr learned my addr 172.21.15.62:0/3449822980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.492+0000 7f7988c09640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7984102c20 0x7f798419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 -- 172.21.15.62:0/3449822980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7984102c20 msgr2=0x7f798419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7984102c20 0x7f798419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 -- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 msgr2=0x7f798419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 -- 172.21.15.62:0/3449822980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798419f360 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f7983fff640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79837fe640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 0x7f798419a8f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800ccc0 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7978010910 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798419f650 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978018960 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.493+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79841a6f30 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.494+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7984100f30 con 0x7f79841022a0 2024-08-16T19:20:57.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.496+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7978010ac0 con 0x7f79841022a0 2024-08-16T19:20:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.496+0000 7f79817fa640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 0x7f796c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.496+0000 7f7983fff640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 0x7f796c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.496+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f79780cffe0 con 0x7f79841022a0 2024-08-16T19:20:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.497+0000 7f7983fff640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 0x7f796c0a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7974011550 tx=0x7f7974005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.500+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f797809c1c0 con 0x7f79841022a0 2024-08-16T19:20:57.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.739+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798419b710 con 0x7f79841022a0 2024-08-16T19:20:57.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.739+0000 7f79817fa640 1 -- 172.21.15.62:0/3449822980 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+12002 (secure 0 0 0) 0x7f798419b710 con 0x7f79841022a0 2024-08-16T19:20:57.740 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:20:57.740 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":56,"btime":"2024-08-16T19:20:39:638044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:20:39.561244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:20:57.741 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 56 2024-08-16T19:20:57.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 msgr2=0x7f796c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 0x7f796c0a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7974011550 tx=0x7f7974005f90 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 msgr2=0x7f798419a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 0x7f798419a8f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 shutdown_connections 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f796c0a2d70 0x7f796c0a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7984102c20 0x7f798419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79841022a0 0x7f798419a8f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 --2- 172.21.15.62:0/3449822980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7984100b50 0x7f798419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 >> 172.21.15.62:0/3449822980 conn(0x7f79840fc500 msgr2=0x7f798406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 shutdown_connections 2024-08-16T19:20:57.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:57.743+0000 7f798a66c640 1 -- 172.21.15.62:0/3449822980 wait complete. 2024-08-16T19:20:57.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 521164 4.67 MB/sec execute 436 sec latency 57.572 ms 2024-08-16T19:20:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:58 smithi062 ceph-mon[32993]: pgmap v325: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:20:58.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:20:58 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3449822980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:20:58.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:58 smithi155 ceph-mon[37413]: pgmap v325: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:20:58.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:20:58 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3449822980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:20:58.637 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0' 2024-08-16T19:20:58.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0 2024-08-16T19:20:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:58 smithi161 ceph-mon[37576]: pgmap v325: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:20:58.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:20:58 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3449822980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:20:58.909 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:20:58.982 INFO:tasks.workunit.client.0.smithi062.stdout: 1 522951 4.69 MB/sec execute 437 sec latency 12.116 ms 2024-08-16T19:20:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 -- 172.21.15.62:0/2433770496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218107fd0 msgr2=0x7f121810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 --2- 172.21.15.62:0/2433770496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218107fd0 0x7f121810e970 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1210009cd0 tx=0x7f121002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 -- 172.21.15.62:0/2433770496 shutdown_connections 2024-08-16T19:20:59.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 --2- 172.21.15.62:0/2433770496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218107fd0 0x7f121810e970 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 --2- 172.21.15.62:0/2433770496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218103410 0x7f12181078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 --2- 172.21.15.62:0/2433770496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218102a60 0x7f1218102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 -- 172.21.15.62:0/2433770496 >> 172.21.15.62:0/2433770496 conn(0x7f12180fc500 msgr2=0x7f12180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 -- 172.21.15.62:0/2433770496 shutdown_connections 2024-08-16T19:20:59.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.384+0000 7f121d09e640 1 -- 172.21.15.62:0/2433770496 wait complete. 2024-08-16T19:20:59.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.385+0000 7f121d09e640 1 Processor -- start 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.385+0000 7f121d09e640 1 -- start start 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.385+0000 7f121d09e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218102a60 0x7f12181006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.385+0000 7f121d09e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121d09e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 0x7f1218101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121d09e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1218111dd0 con 0x7f1218103410 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121d09e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1218111c50 con 0x7f1218107fd0 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121d09e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1218111f50 con 0x7f1218102a60 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49118/0 (socket says 172.21.15.62:49118) 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 -- 172.21.15.62:0/3152534728 learned_addr learned my addr 172.21.15.62:0/3152534728 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:59.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121759e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 0x7f1218101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 -- 172.21.15.62:0/3152534728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218102a60 msgr2=0x7f12181006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218102a60 0x7f12181006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 -- 172.21.15.62:0/3152534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 msgr2=0x7f1218101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 0x7f1218101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 -- 172.21.15.62:0/3152534728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12181a67e0 con 0x7f1218103410 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121759e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 0x7f1218101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f121659c640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9f0 tx=0x7f120c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.386+0000 7f11fffff640 1 -- 172.21.15.62:0/3152534728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00ccc0 con 0x7f1218103410 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.387+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12181a6ad0 con 0x7f1218103410 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.387+0000 7f11fffff640 1 -- 172.21.15.62:0/3152534728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c010910 con 0x7f1218103410 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.387+0000 7f11fffff640 1 -- 172.21.15.62:0/3152534728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c018920 con 0x7f1218103410 2024-08-16T19:20:59.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.387+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181a7060 con 0x7f1218103410 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 Processor -- start 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f11fffff640 1 -- 172.21.15.62:0/3152534728 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f120c018ac0 con 0x7f1218103410 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 -- start start 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf652890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650bf5aa810 con 0x5650bf6504a0 2024-08-16T19:20:59.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650bf551c70 con 0x5650bf645be0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.389+0000 7f121f49c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650bf5a67d0 con 0x5650bf6452c0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:56162/0 (socket says 172.21.15.62:56162) 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:49484/0 (socket says 172.21.15.62:49484) 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f1216d9d640 1 -- 172.21.15.62:0/833259084 learned_addr learned my addr 172.21.15.62:0/833259084 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121659c640 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf652890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 -- 172.21.15.62:0/833259084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 msgr2=0x5650bf64ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 -- 172.21.15.62:0/833259084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 msgr2=0x5650bf652890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf652890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 -- 172.21.15.62:0/833259084 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bf31cba0 con 0x5650bf6452c0 2024-08-16T19:20:59.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f1216d9d640 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f121759e640 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1210009ca0 tx=0x7f1210009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1f2bbaf465b054 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f11fd7fa640 1 -- 172.21.15.62:0/833259084 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210048070 con 0x5650bf6452c0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.390+0000 7f11fd7fa640 1 -- 172.21.15.62:0/833259084 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1210036070 con 0x5650bf6452c0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f11fffff640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 0x7f12000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f11fd7fa640 1 -- 172.21.15.62:0/833259084 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210004660 con 0x5650bf6452c0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f1216d9d640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 0x7f12000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f11fffff640 1 -- 172.21.15.62:0/3152534728 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f120c0cfb60 con 0x7f1218103410 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 -- 172.21.15.62:0/833259084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 msgr2=0x5650bf6456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f1216d9d640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 0x7f12000a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5650bf6426b0 tx=0x7f1208009b30 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1210009ca0 tx=0x7f1210009a20 comp rx=0 tx=0).stop 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 -- 172.21.15.62:0/833259084 shutdown_connections 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf652890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf64ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 --2- 172.21.15.62:0/833259084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf6452c0 0x5650bf6456a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 -- 172.21.15.62:0/833259084 >> 172.21.15.62:0/833259084 conn(0x5650bf641850 msgr2=0x5650bf641c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 -- 172.21.15.62:0/833259084 shutdown_connections 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.391+0000 7f121f49c740 1 -- 172.21.15.62:0/833259084 wait complete. 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 -- start start 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 Processor -- start 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf64da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf64df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5650bf652fb0 con 0x5650bf6452c0 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5650bf5659a0 con 0x5650bf6504a0 2024-08-16T19:20:59.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5650bf653130 con 0x5650bf645be0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121f49c740 10 client.0 _ll_register_callbacks cb 0x5650bf553c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf64da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49134/0 (socket says 172.21.15.62:49134) 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f1216d9d640 1 -- 172.21.15.62:0/4041892649 learned_addr learned my addr 172.21.15.62:0/4041892649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.392+0000 7f121759e640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf64df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 -- 172.21.15.62:0/4041892649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 msgr2=0x5650bf64da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf64da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 -- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 msgr2=0x5650bf64df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf64df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 -- 172.21.15.62:0/4041892649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bf571220 con 0x5650bf6452c0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f1216d9d640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1208009ac0 tx=0x7f12080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42b3b48b3d8441a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f11faff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:20:59.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120801e830 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1210036070 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f120801e9d0 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f120801ebd0 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f121f49c740 10 client.15972 fetch_fsmap learned FSMap version 56 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.393+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5650bf65eda0 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1210036070 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f120801c3b0 con 0x5650bf6452c0 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 10 client.15972 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 10 client.15972 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 20 client.15972 populate_metadata read hostname 'smithi062' 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf660250 0x5650bf662660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 4 client.15972 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bf5b7f40 con 0x5650bf660250 2024-08-16T19:20:59.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121659c640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf660250 0x5650bf662660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f11faff5640 10 client.15972 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.394+0000 7f121659c640 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf660250 0x5650bf662660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea2e0cf142146b0 server_cookie=ee61b56be8187be0 in_seq=0 out_seq=0 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f11faff5640 1 -- 172.21.15.62:0/4041892649 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650bf5b7f40 con 0x5650bf660250 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f11faff5640 10 client.15972 handle_command_reply: tid=0 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 client.15972 shutdown 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 msgr2=0x5650bf64d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1208009ac0 tx=0x7f12080046c0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 shutdown_connections 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf660250 0x5650bf662660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf64df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f11faff5640 0 client.15972 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf64da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 --2- 172.21.15.62:0/4041892649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6452c0 0x5650bf64d4f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 >> 172.21.15.62:0/4041892649 conn(0x5650bf641850 msgr2=0x5650bf640b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 shutdown_connections 2024-08-16T19:20:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.395+0000 7f121f49c740 1 -- 172.21.15.62:0/4041892649 wait complete. 2024-08-16T19:20:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.396+0000 7f121f49c740 20 client.15972 trim_cache size 0 max 16384 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 Processor -- start 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 -- start start 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf667f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf6684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12181b1130 con 0x5650bf64a750 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12181b2d30 con 0x5650bf6504a0 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121f49c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12181b1800 con 0x5650bf645be0 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f121759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf6684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.413+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf667f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49142/0 (socket says 172.21.15.62:49142) 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf667f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:49492/0 (socket says 172.21.15.62:49492) 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121759e640 1 -- 172.21.15.62:0/3895901915 learned_addr learned my addr 172.21.15.62:0/3895901915 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 -- 172.21.15.62:0/3895901915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 msgr2=0x5650bf667f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf667f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 -- 172.21.15.62:0/3895901915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 msgr2=0x5650bf64abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 -- 172.21.15.62:0/3895901915 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bf64c4a0 con 0x5650bf6504a0 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121759e640 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f121659c640 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf6684b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f120c0ccec0 tx=0x7f120c0d36b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cdd2a0c6474e926 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f11fcff9640 1 -- 172.21.15.62:0/3895901915 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c09c390 con 0x5650bf6504a0 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f11fcff9640 1 -- 172.21.15.62:0/3895901915 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c09bc90 con 0x5650bf6504a0 2024-08-16T19:20:59.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.414+0000 7f11fcff9640 1 -- 172.21.15.62:0/3895901915 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c010ac0 con 0x5650bf6504a0 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- 172.21.15.62:0/3895901915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 msgr2=0x5650bf6684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf6684b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f120c0ccec0 tx=0x7f120c0d36b0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- 172.21.15.62:0/3895901915 shutdown_connections 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf64a750 0x5650bf64abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf6504a0 0x5650bf6684b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 --2- 172.21.15.62:0/3895901915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf645be0 0x5650bf667f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- 172.21.15.62:0/3895901915 >> 172.21.15.62:0/3895901915 conn(0x5650bf641850 msgr2=0x5650bf640b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- 172.21.15.62:0/3895901915 shutdown_connections 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- 172.21.15.62:0/3895901915 wait complete. 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.415+0000 7f121f49c740 1 -- start start 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 Processor -- start 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf658690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 0x5650bf658bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12181b25a0 con 0x5650bf6504a0 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f12181b1ed0 con 0x5650bf645be0 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f12181af0d0 con 0x5650bf64a750 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121f49c740 10 client.0 _ll_register_callbacks cb 0x7f1208004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 0x5650bf658bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f1216d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf658690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49144/0 (socket says 172.21.15.62:49144) 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 0x5650bf658bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:49502/0 (socket says 172.21.15.62:49502) 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 -- 172.21.15.62:0/2604062972 learned_addr learned my addr 172.21.15.62:0/2604062972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 -- 172.21.15.62:0/2604062972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 msgr2=0x5650bf658bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 0x5650bf658bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 -- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 msgr2=0x5650bf658690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf658690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 -- 172.21.15.62:0/2604062972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bf655c30 con 0x5650bf6504a0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f121759e640 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1210006010 tx=0x7f121002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5086a924306aaa server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.416+0000 7f11faff5640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210054070 con 0x5650bf6504a0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f12100046e0 con 0x5650bf6504a0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c09bc90 con 0x5650bf6504a0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f1210048040 con 0x5650bf6504a0 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 10 client.15978 fetch_fsmap learned FSMap version 56 2024-08-16T19:20:59.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5650bf6cbc10 con 0x5650bf6504a0 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c09bc90 con 0x5650bf6504a0 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f1210042030 con 0x5650bf6504a0 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 10 client.15978 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 10 client.15978 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 20 client.15978 populate_metadata read hostname 'smithi062' 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf6cc120 0x5650bf6ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 4 client.15978 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}] 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.417+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}) -- 0x5650bf5b2290 con 0x5650bf6cc120 2024-08-16T19:20:59.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.418+0000 7f1216d9d640 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf6cc120 0x5650bf6ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:20:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.418+0000 7f1216d9d640 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf6cc120 0x5650bf6ce510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ae0090b69a2f29 server_cookie=eefce8dd80486fce in_seq=0 out_seq=0 2024-08-16T19:20:59.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.418+0000 7f11faff5640 10 client.15978 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:20:59.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f11faff5640 1 -- 172.21.15.62:0/2604062972 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650bf5b2290 con 0x5650bf6cc120 2024-08-16T19:20:59.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f11faff5640 10 client.15978 handle_command_reply: tid=0 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 client.15978 shutdown 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 msgr2=0x5650bf655430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1210006010 tx=0x7f121002a8e0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 shutdown_connections 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5650bf6cc120 0x5650bf6ce510 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5650bf6504a0 0x5650bf655430 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650bf64a750 0x5650bf658bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 --2- 172.21.15.62:0/2604062972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650bf645be0 0x5650bf658690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 >> 172.21.15.62:0/2604062972 conn(0x5650bf641850 msgr2=0x5650bf640b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f11faff5640 0 client.15978 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:20:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 shutdown_connections 2024-08-16T19:20:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 1 -- 172.21.15.62:0/2604062972 wait complete. 2024-08-16T19:20:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.486+0000 7f121f49c740 20 client.15978 trim_cache size 0 max 16384 2024-08-16T19:20:59.489 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 56, 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 38, 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "1b610241": { 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 38, 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:20:59.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.9, 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:20:59.491 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:20:59.492 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 msgr2=0x7f12000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 0x7f12000a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5650bf6426b0 tx=0x7f1208009b30 comp rx=0 tx=0).stop 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 msgr2=0x7f1218100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f120c00e9f0 tx=0x7f120c00eeb0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 shutdown_connections 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f12000a2cf0 0x7f12000a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1218107fd0 0x7f1218101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1218103410 0x7f1218100c00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 --2- 172.21.15.62:0/3152534728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1218102a60 0x7f12181006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:20:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 >> 172.21.15.62:0/3152534728 conn(0x7f12180fc500 msgr2=0x7f1218105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:20:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 shutdown_connections 2024-08-16T19:20:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:20:59.487+0000 7f121d09e640 1 -- 172.21.15.62:0/3152534728 wait complete. 2024-08-16T19:20:59.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.69 MB/sec execute 438 sec latency 489.854 ms 2024-08-16T19:21:00.198 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1b610241', quiesce took 3.5 seconds. Will release after: 24.45 2024-08-16T19:21:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:00 smithi062 ceph-mon[32993]: pgmap v326: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:21:00.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:00 smithi155 ceph-mon[37413]: pgmap v326: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:21:00.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:00 smithi161 ceph-mon[37576]: pgmap v326: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 202 op/s 2024-08-16T19:21:00.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.68 MB/sec execute 439 sec latency 1490.315 ms 2024-08-16T19:21:01.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.67 MB/sec execute 440 sec latency 2490.418 ms 2024-08-16T19:21:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:02 smithi062 ceph-mon[32993]: pgmap v327: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-08-16T19:21:02.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:02 smithi155 ceph-mon[37413]: pgmap v327: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-08-16T19:21:02.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:02 smithi161 ceph-mon[37576]: pgmap v327: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 311 op/s 2024-08-16T19:21:02.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.66 MB/sec execute 441 sec latency 3490.537 ms 2024-08-16T19:21:03.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.64 MB/sec execute 442 sec latency 4490.599 ms 2024-08-16T19:21:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:04 smithi062 ceph-mon[32993]: pgmap v328: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-08-16T19:21:04.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:04 smithi155 ceph-mon[37413]: pgmap v328: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-08-16T19:21:04.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:04 smithi161 ceph-mon[37576]: pgmap v328: 129 pgs: 129 active+clean; 423 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-08-16T19:21:04.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.63 MB/sec execute 443 sec latency 5490.703 ms 2024-08-16T19:21:05.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.62 MB/sec execute 444 sec latency 6490.843 ms 2024-08-16T19:21:06.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:06 smithi062 ceph-mon[32993]: pgmap v329: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-08-16T19:21:06.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:06 smithi062 ceph-mon[32993]: osdmap e102: 12 total, 12 up, 12 in 2024-08-16T19:21:06.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:06 smithi155 ceph-mon[37413]: pgmap v329: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-08-16T19:21:06.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:06 smithi155 ceph-mon[37413]: osdmap e102: 12 total, 12 up, 12 in 2024-08-16T19:21:06.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:06 smithi161 ceph-mon[37576]: pgmap v329: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-08-16T19:21:06.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:06 smithi161 ceph-mon[37576]: osdmap e102: 12 total, 12 up, 12 in 2024-08-16T19:21:06.983 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.61 MB/sec execute 445 sec latency 7490.967 ms 2024-08-16T19:21:07.830 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:07 smithi155 ceph-mon[37413]: pgmap v331: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:07.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:07 smithi161 ceph-mon[37576]: pgmap v331: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:07.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.60 MB/sec execute 446 sec latency 8491.059 ms 2024-08-16T19:21:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:07 smithi062 ceph-mon[32993]: pgmap v331: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:08.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.59 MB/sec execute 447 sec latency 9491.180 ms 2024-08-16T19:21:09.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:09 smithi155 ceph-mon[37413]: pgmap v332: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:09.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:09 smithi161 ceph-mon[37576]: pgmap v332: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:09.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.58 MB/sec execute 448 sec latency 10491.285 ms 2024-08-16T19:21:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:09 smithi062 ceph-mon[32993]: pgmap v332: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-08-16T19:21:10.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.57 MB/sec execute 449 sec latency 11491.394 ms 2024-08-16T19:21:11.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:11 smithi155 ceph-mon[37413]: pgmap v333: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-08-16T19:21:11.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:11 smithi161 ceph-mon[37576]: pgmap v333: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-08-16T19:21:11.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.56 MB/sec execute 450 sec latency 12491.464 ms 2024-08-16T19:21:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:11 smithi062 ceph-mon[32993]: pgmap v333: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-08-16T19:21:12.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:12 smithi161 ceph-mon[37576]: osdmap e103: 12 total, 12 up, 12 in 2024-08-16T19:21:12.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.55 MB/sec execute 451 sec latency 13491.561 ms 2024-08-16T19:21:12.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:12 smithi062 ceph-mon[32993]: osdmap e103: 12 total, 12 up, 12 in 2024-08-16T19:21:13.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:12 smithi155 ceph-mon[37413]: osdmap e103: 12 total, 12 up, 12 in 2024-08-16T19:21:13.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:13 smithi161 ceph-mon[37576]: pgmap v335: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 818 KiB/s wr, 36 op/s 2024-08-16T19:21:13.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.54 MB/sec execute 452 sec latency 14491.615 ms 2024-08-16T19:21:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:13 smithi062 ceph-mon[32993]: pgmap v335: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 818 KiB/s wr, 36 op/s 2024-08-16T19:21:14.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:13 smithi155 ceph-mon[37413]: pgmap v335: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 818 KiB/s wr, 36 op/s 2024-08-16T19:21:14.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.53 MB/sec execute 453 sec latency 15491.665 ms 2024-08-16T19:21:15.881 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:15 smithi155 ceph-mon[37413]: pgmap v336: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.6 MiB/s wr, 69 op/s 2024-08-16T19:21:15.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:15 smithi161 ceph-mon[37576]: pgmap v336: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.6 MiB/s wr, 69 op/s 2024-08-16T19:21:15.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.52 MB/sec execute 454 sec latency 16491.722 ms 2024-08-16T19:21:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:15 smithi062 ceph-mon[32993]: pgmap v336: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.6 MiB/s wr, 69 op/s 2024-08-16T19:21:16.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.51 MB/sec execute 455 sec latency 17491.820 ms 2024-08-16T19:21:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:17 smithi161 ceph-mon[37576]: pgmap v337: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:17.984 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.50 MB/sec execute 456 sec latency 18491.896 ms 2024-08-16T19:21:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:17 smithi062 ceph-mon[32993]: pgmap v337: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:17 smithi155 ceph-mon[37413]: pgmap v337: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:18.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.49 MB/sec execute 457 sec latency 19492.031 ms 2024-08-16T19:21:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:19 smithi161 ceph-mon[37576]: pgmap v338: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:19.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.48 MB/sec execute 458 sec latency 20492.154 ms 2024-08-16T19:21:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:19 smithi062 ceph-mon[32993]: pgmap v338: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:19 smithi155 ceph-mon[37413]: pgmap v338: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 64 op/s 2024-08-16T19:21:20.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.47 MB/sec execute 459 sec latency 21492.288 ms 2024-08-16T19:21:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:21 smithi161 ceph-mon[37576]: pgmap v339: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-08-16T19:21:21.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.46 MB/sec execute 460 sec latency 22492.403 ms 2024-08-16T19:21:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:21 smithi062 ceph-mon[32993]: pgmap v339: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-08-16T19:21:22.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:21 smithi155 ceph-mon[37413]: pgmap v339: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-08-16T19:21:22.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.45 MB/sec execute 461 sec latency 23492.525 ms 2024-08-16T19:21:23.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:23 smithi161 ceph-mon[37576]: pgmap v340: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-08-16T19:21:23.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.44 MB/sec execute 462 sec latency 24492.626 ms 2024-08-16T19:21:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:23 smithi062 ceph-mon[32993]: pgmap v340: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-08-16T19:21:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:23 smithi155 ceph-mon[37413]: pgmap v340: 129 pgs: 129 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-08-16T19:21:24.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:21:24.929 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:25.009 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.43 MB/sec execute 463 sec latency 25492.716 ms 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 -- 172.21.15.62:0/3360975100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c102c20 msgr2=0x7fae5c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 --2- 172.21.15.62:0/3360975100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c102c20 0x7fae5c10d770 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae54009ca0 tx=0x7fae5402a950 comp rx=0 tx=0).stop 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 -- 172.21.15.62:0/3360975100 shutdown_connections 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 --2- 172.21.15.62:0/3360975100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c102c20 0x7fae5c10d770 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 --2- 172.21.15.62:0/3360975100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 0x7fae5c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 --2- 172.21.15.62:0/3360975100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c100b50 0x7fae5c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 -- 172.21.15.62:0/3360975100 >> 172.21.15.62:0/3360975100 conn(0x7fae5c0fc500 msgr2=0x7fae5c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:25.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 -- 172.21.15.62:0/3360975100 shutdown_connections 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.389+0000 7fae62a57640 1 -- 172.21.15.62:0/3360975100 wait complete. 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.390+0000 7fae62a57640 1 Processor -- start 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.390+0000 7fae62a57640 1 -- start start 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.390+0000 7fae62a57640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.390+0000 7fae62a57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 0x7fae5c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.390+0000 7fae62a57640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c102c20 0x7fae5c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae62a57640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110cb0 con 0x7fae5c100b50 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae62a57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110b30 con 0x7fae5c1022a0 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae62a57640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fae5c110e30 con 0x7fae5c102c20 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53580/0 (socket says 172.21.15.62:53580) 2024-08-16T19:21:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 -- 172.21.15.62:0/3391623714 learned_addr learned my addr 172.21.15.62:0/3391623714 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae60ff4640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c102c20 0x7fae5c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae53fff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 0x7fae5c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 -- 172.21.15.62:0/3391623714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c102c20 msgr2=0x7fae5c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c102c20 0x7fae5c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 -- 172.21.15.62:0/3391623714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 msgr2=0x7fae5c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 0x7fae5c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 -- 172.21.15.62:0/3391623714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5c19f360 con 0x7fae5c100b50 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae5bfff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00d770 tx=0x7fae4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c017070 con 0x7fae5c100b50 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c005500 con 0x7fae5c100b50 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5c19f650 con 0x7fae5c100b50 2024-08-16T19:21:25.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004e00 con 0x7fae5c100b50 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.391+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5c1a6f30 con 0x7fae5c100b50 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.392+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5c100f30 con 0x7fae5c100b50 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.394+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae4c004fa0 con 0x7fae5c100b50 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.394+0000 7fae59ffb640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 0x7fae480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.394+0000 7fae53fff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 0x7fae480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.394+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fae4c0cf370 con 0x7fae5c100b50 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.395+0000 7fae53fff640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 0x7fae480a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae44005fd0 tx=0x7fae44009470 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:25.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.399+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fae4c09bba0 con 0x7fae5c100b50 2024-08-16T19:21:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.612+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae5c1a7150 con 0x7fae5c100b50 2024-08-16T19:21:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.613+0000 7fae59ffb640 1 -- 172.21.15.62:0/3391623714 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+12002 (secure 0 0 0) 0x7fae4c09b4a0 con 0x7fae5c100b50 2024-08-16T19:21:25.616 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:21:25.616 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":56,"btime":"2024-08-16T19:20:39:638044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:20:39.561244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 56 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 msgr2=0x7fae480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 0x7fae480a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae44005fd0 tx=0x7fae44009470 comp rx=0 tx=0).stop 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 msgr2=0x7fae5c19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00d770 tx=0x7fae4c00dc30 comp rx=0 tx=0).stop 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 shutdown_connections 2024-08-16T19:21:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae480a2cf0 0x7fae480a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae5c102c20 0x7fae5c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5c1022a0 0x7fae5c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 --2- 172.21.15.62:0/3391623714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae5c100b50 0x7fae5c19a3b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 >> 172.21.15.62:0/3391623714 conn(0x7fae5c0fc500 msgr2=0x7fae5c069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 shutdown_connections 2024-08-16T19:21:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:25.615+0000 7fae62a57640 1 -- 172.21.15.62:0/3391623714 wait complete. 2024-08-16T19:21:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:25 smithi161 ceph-mon[37576]: pgmap v341: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-08-16T19:21:25.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.42 MB/sec execute 464 sec latency 26492.785 ms 2024-08-16T19:21:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:25 smithi062 ceph-mon[32993]: pgmap v341: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-08-16T19:21:26.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:25 smithi155 ceph-mon[37413]: pgmap v341: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-08-16T19:21:26.460 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 1b610241 --release --await' 2024-08-16T19:21:26.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 1b610241 --release --await 2024-08-16T19:21:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:26.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:26 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3391623714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:26.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:26 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3391623714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:26.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 523685 4.42 MB/sec execute 465 sec latency 27492.812 ms 2024-08-16T19:21:27.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:26 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3391623714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:27.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 -- 172.21.15.62:0/4251382204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 msgr2=0x7f094010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 --2- 172.21.15.62:0/4251382204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094010d770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0938009900 tx=0x7f093802a480 comp rx=0 tx=0).stop 2024-08-16T19:21:27.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 -- 172.21.15.62:0/4251382204 shutdown_connections 2024-08-16T19:21:27.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 --2- 172.21.15.62:0/4251382204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094010d770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 --2- 172.21.15.62:0/4251382204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 0x7f09401026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 --2- 172.21.15.62:0/4251382204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f0940100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 -- 172.21.15.62:0/4251382204 >> 172.21.15.62:0/4251382204 conn(0x7f09400fc500 msgr2=0x7f09400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 -- 172.21.15.62:0/4251382204 shutdown_connections 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.151+0000 7f094817e640 1 -- 172.21.15.62:0/4251382204 wait complete. 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 Processor -- start 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 -- start start 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f094019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 0x7f094019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0940110cb0 con 0x7f0940102c20 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0940110b30 con 0x7f0940100b50 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094817e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0940110e30 con 0x7f09401022a0 2024-08-16T19:21:27.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094671b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f0945f1a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f094019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094671b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53600/0 (socket says 172.21.15.62:53600) 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f094671b640 1 -- 172.21.15.62:0/2014265205 learned_addr learned my addr 172.21.15.62:0/2014265205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.152+0000 7f0945719640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 0x7f094019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 -- 172.21.15.62:0/2014265205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 msgr2=0x7f094019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 0x7f094019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 -- 172.21.15.62:0/2014265205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 msgr2=0x7f094019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f094019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 -- 172.21.15.62:0/2014265205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094019f2c0 con 0x7f0940102c20 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f0945f1a640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f094019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094671b640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0938004770 tx=0x7f0938003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f093effd640 1 -- 172.21.15.62:0/2014265205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093803f070 con 0x7f0940102c20 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094019f550 con 0x7f0940102c20 2024-08-16T19:21:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f093effd640 1 -- 172.21.15.62:0/2014265205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0938031c30 con 0x7f0940102c20 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f093effd640 1 -- 172.21.15.62:0/2014265205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938035030 con 0x7f0940102c20 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.153+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09401a6e90 con 0x7f0940102c20 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f094a57c740 1 Processor -- start 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f093effd640 1 -- 172.21.15.62:0/2014265205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0938042070 con 0x7f0940102c20 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f094a57c740 1 -- start start 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f094a57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x56189302cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f094a57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.156+0000 7f094a57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893039f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f094a57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561892f92c60 con 0x56189302d2b0 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f094a57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561892f2ab90 con 0x56189302c990 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f094a57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561892f8ec20 con 0x561893037b70 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53616/0 (socket says 172.21.15.62:53616) 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 -- 172.21.15.62:0/2987329509 learned_addr learned my addr 172.21.15.62:0/2987329509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945719640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893039f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f093effd640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 0x7f09280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945719640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x56189302cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 -- 172.21.15.62:0/2987329509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 msgr2=0x561893039f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893039f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 -- 172.21.15.62:0/2987329509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 msgr2=0x56189302cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x56189302cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 -- 172.21.15.62:0/2987329509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561892ca2580 con 0x56189302d2b0 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945719640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893039f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f094671b640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 0x7f09280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f093effd640 1 -- 172.21.15.62:0/2014265205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f09380f0320 con 0x7f0940102c20 2024-08-16T19:21:27.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.157+0000 7f0945f1a640 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f093000f4c0 tx=0x7f093000f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1686dda70668e008 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f0927fff640 1 -- 172.21.15.62:0/2987329509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930005220 con 0x56189302d2b0 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f0927fff640 1 -- 172.21.15.62:0/2987329509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930004c80 con 0x56189302d2b0 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f0927fff640 1 -- 172.21.15.62:0/2987329509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930010c20 con 0x56189302d2b0 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 -- 172.21.15.62:0/2987329509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 msgr2=0x561893037630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094671b640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 0x7f09280a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09380bc3e0 tx=0x7f0938096040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f093000f4c0 tx=0x7f093000f980 comp rx=0 tx=0).stop 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 -- 172.21.15.62:0/2987329509 shutdown_connections 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893039f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893037630 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 --2- 172.21.15.62:0/2987329509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x56189302cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.158+0000 7f094a57c740 1 -- 172.21.15.62:0/2987329509 >> 172.21.15.62:0/2987329509 conn(0x561893028f20 msgr2=0x561893029300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- 172.21.15.62:0/2987329509 shutdown_connections 2024-08-16T19:21:27.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- 172.21.15.62:0/2987329509 wait complete. 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- start start 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 Processor -- start 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893035430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56189303a740 con 0x56189302d2b0 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561892f307c0 con 0x56189302c990 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56189303a8c0 con 0x561893037b70 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.159+0000 7f094a57c740 10 client.0 _ll_register_callbacks cb 0x561892f39f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53632/0 (socket says 172.21.15.62:53632) 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 -- 172.21.15.62:0/1373240147 learned_addr learned my addr 172.21.15.62:0/1373240147 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945719640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893035430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945719640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 -- 172.21.15.62:0/1373240147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 msgr2=0x561893035430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893035430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 -- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 msgr2=0x5618930349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 -- 172.21.15.62:0/1373240147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561892d6c960 con 0x56189302d2b0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945719640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f0945f1a640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f09300107a0 tx=0x7f0930004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af983283a3f6998 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f09257fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.160+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930026070 con 0x56189302d2b0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0930004c80 con 0x56189302d2b0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0930010db0 con 0x56189302d2b0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f0930032050 con 0x56189302d2b0 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 10 client.15996 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:27.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x561893046700 con 0x56189302d2b0 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0930004c80 con 0x56189302d2b0 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f0930018bf0 con 0x56189302d2b0 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 10 client.15996 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 10 client.15996 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 20 client.15996 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561893047b10 0x561893049f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 4 client.15996 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.161+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561892fa01d0 con 0x561893047b10 2024-08-16T19:21:27.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.162+0000 7f0945719640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561893047b10 0x561893049f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.162+0000 7f0945719640 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561893047b10 0x561893049f00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b85fefec638f32 server_cookie=fb38b63ea3ac8aa in_seq=0 out_seq=0 2024-08-16T19:21:27.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.162+0000 7f09257fa640 10 client.15996 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:27.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f09257fa640 1 -- 172.21.15.62:0/1373240147 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561892fa01d0 con 0x561893047b10 2024-08-16T19:21:27.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f09257fa640 10 client.15996 handle_command_reply: tid=0 2024-08-16T19:21:27.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 client.15996 shutdown 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 msgr2=0x561893034ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f09300107a0 tx=0x7f0930004970 comp rx=0 tx=0).stop 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 shutdown_connections 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561893047b10 0x561893049f00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561893037b70 0x561893035430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302d2b0 0x561893034ef0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 --2- 172.21.15.62:0/1373240147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f09257fa640 0 client.15996 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 >> 172.21.15.62:0/1373240147 conn(0x561893028f20 msgr2=0x561893028080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.163+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 shutdown_connections 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.164+0000 7f094a57c740 1 -- 172.21.15.62:0/1373240147 wait complete. 2024-08-16T19:21:27.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.164+0000 7f094a57c740 20 client.15996 trim_cache size 0 max 16384 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 Processor -- start 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 -- start start 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189304f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561893037b70 0x56189304fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b1030 con 0x561893037b70 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b2c30 con 0x56189302c990 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f094a57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b1700 con 0x56189302d2b0 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f0945719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561893037b70 0x56189304fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f0945719640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f0945719640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:56624/0 (socket says 172.21.15.62:56624) 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f0945719640 1 -- 172.21.15.62:0/314853920 learned_addr learned my addr 172.21.15.62:0/314853920 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.181+0000 7f0945f1a640 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189304f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 -- 172.21.15.62:0/314853920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 msgr2=0x56189304f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189304f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 -- 172.21.15.62:0/314853920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561893037b70 msgr2=0x56189304fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561893037b70 0x56189304fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 -- 172.21.15.62:0/314853920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56189302d0e0 con 0x56189302c990 2024-08-16T19:21:27.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f0945719640 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0934009fd0 tx=0x7f0934009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cdabb2f0504f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f09277fe640 1 -- 172.21.15.62:0/314853920 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093401b040 con 0x56189302c990 2024-08-16T19:21:27.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f09277fe640 1 -- 172.21.15.62:0/314853920 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0934026040 con 0x56189302c990 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.182+0000 7f09277fe640 1 -- 172.21.15.62:0/314853920 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934010ca0 con 0x56189302c990 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- 172.21.15.62:0/314853920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 msgr2=0x5618930338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0934009fd0 tx=0x7f0934009680 comp rx=0 tx=0).stop 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- 172.21.15.62:0/314853920 shutdown_connections 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561893037b70 0x56189304fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189304f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 --2- 172.21.15.62:0/314853920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56189302c990 0x5618930338f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- 172.21.15.62:0/314853920 >> 172.21.15.62:0/314853920 conn(0x561893028f20 msgr2=0x561893036a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- 172.21.15.62:0/314853920 shutdown_connections 2024-08-16T19:21:27.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- 172.21.15.62:0/314853920 wait complete. 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 -- start start 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.183+0000 7f094a57c740 1 Processor -- start 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189303bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561893037b70 0x56189303c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b24a0 con 0x56189302c990 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f09401b1dd0 con 0x561893037b70 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f09401aefd0 con 0x56189302d2b0 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f094a57c740 10 client.0 _ll_register_callbacks cb 0x7f09300053c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53650/0 (socket says 172.21.15.62:53650) 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 -- 172.21.15.62:0/174499497 learned_addr learned my addr 172.21.15.62:0/174499497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945f1a640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189303bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 -- 172.21.15.62:0/174499497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 msgr2=0x56189303bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189303bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 -- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561893037b70 msgr2=0x56189303c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561893037b70 0x56189303c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.184+0000 7f0945719640 1 -- 172.21.15.62:0/174499497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618930b1770 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f0945719640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f0934024450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa457738c422156 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934004b20 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0934004cc0 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0934026040 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f0934010a20 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f094a57c740 10 client.16002 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5618930b2a70 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0934026040 con 0x56189302c990 2024-08-16T19:21:27.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.185+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f093402b040 con 0x56189302c990 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 10 client.16002 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 10 client.16002 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 20 client.16002 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5618930b3190 0x5618930b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 4 client.16002 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1b610241", "release": true, "await": true}] 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1b610241", "release": true, "await": true}) -- 0x561892f9a6e0 con 0x5618930b3190 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f0945f1a640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5618930b3190 0x5618930b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f0945f1a640 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5618930b3190 0x5618930b55b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bc80751e3b558b server_cookie=118d5ea06a49cf8d in_seq=0 out_seq=0 2024-08-16T19:21:27.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.186+0000 7f09257fa640 10 client.16002 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f09257fa640 1 -- 172.21.15.62:0/174499497 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561892f9a6e0 con 0x5618930b3190 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f09257fa640 10 client.16002 handle_command_reply: tid=0 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 client.16002 shutdown 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 msgr2=0x56189303b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f0934024450 comp rx=0 tx=0).stop 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 shutdown_connections 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5618930b3190 0x5618930b55b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f09257fa640 0 client.16002 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:27.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561893037b70 0x56189303c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56189302d2b0 0x56189303bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 --2- 172.21.15.62:0/174499497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56189302c990 0x56189303b9f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 >> 172.21.15.62:0/174499497 conn(0x561893028f20 msgr2=0x561893027860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 shutdown_connections 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 1 -- 172.21.15.62:0/174499497 wait complete. 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.197+0000 7f094a57c740 20 client.16002 trim_cache size 0 max 16384 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 56, 2024-08-16T19:21:27.199 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 40, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "1b610241": { 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 40, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.9, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:21:27.200 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:27.201 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 msgr2=0x7f09280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 0x7f09280a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09380bc3e0 tx=0x7f0938096040 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 msgr2=0x7f094019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0938004770 tx=0x7f0938003c10 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 shutdown_connections 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f09280a2cf0 0x7f09280a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0940102c20 0x7f094019ebe0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09401022a0 0x7f094019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 --2- 172.21.15.62:0/2014265205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0940100b50 0x7f094019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:27.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 >> 172.21.15.62:0/2014265205 conn(0x7f09400fc500 msgr2=0x7f094006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:27.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 shutdown_connections 2024-08-16T19:21:27.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:27.198+0000 7f094817e640 1 -- 172.21.15.62:0/2014265205 wait complete. 2024-08-16T19:21:27.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:27 smithi062 ceph-mon[32993]: pgmap v342: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:27.886 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1b610241', total seconds elapsed: 31.2 2024-08-16T19:21:27.886 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.95 2024-08-16T19:21:27.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:21:27.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:27 smithi161 ceph-mon[37576]: pgmap v342: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:27.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 525402 4.42 MB/sec execute 466 sec latency 27706.796 ms 2024-08-16T19:21:28.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:27 smithi155 ceph-mon[37413]: pgmap v342: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:28.211 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 -- 172.21.15.62:0/552657751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc10bea0 msgr2=0x7f1bbc10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/552657751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc10e340 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009ca0 tx=0x7f1bb402a950 comp rx=0 tx=0).stop 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 -- 172.21.15.62:0/552657751 shutdown_connections 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/552657751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc10e340 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/552657751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 0x7f1bbc10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/552657751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102a70 0x7f1bbc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 -- 172.21.15.62:0/552657751 >> 172.21.15.62:0/552657751 conn(0x7f1bbc0fc4d0 msgr2=0x7f1bbc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 -- 172.21.15.62:0/552657751 shutdown_connections 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.677+0000 7f1bc39c6640 1 -- 172.21.15.62:0/552657751 wait complete. 2024-08-16T19:21:28.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 Processor -- start 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 -- start start 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 0x7f1bbc198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc111ca0 con 0x7f1bbc102a70 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc111b20 con 0x7f1bbc10bea0 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc39c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc111e20 con 0x7f1bbc103390 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc1762640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc1762640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53672/0 (socket says 172.21.15.62:53672) 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc1762640 1 -- 172.21.15.62:0/3166046989 learned_addr learned my addr 172.21.15.62:0/3166046989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc0f61640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 0x7f1bbc198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.678+0000 7f1bc1f63640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:28.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 -- 172.21.15.62:0/3166046989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 msgr2=0x7f1bbc198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 0x7f1bbc198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 -- 172.21.15.62:0/3166046989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc10bea0 msgr2=0x7f1bbc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 -- 172.21.15.62:0/3166046989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc19d0b0 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc1762640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e9f0 tx=0x7f1bb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800cd80 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bb8010910 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc19d3a0 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8018990 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.679+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1a4c80 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.680+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc102e50 con 0x7f1bbc102a70 2024-08-16T19:21:28.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.681+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1bb8010ac0 con 0x7f1bbc102a70 2024-08-16T19:21:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.682+0000 7f1bb27fc640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 0x7f1ba80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.682+0000 7f1bc0f61640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 0x7f1ba80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.682+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f1bb80cfed0 con 0x7f1bbc102a70 2024-08-16T19:21:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.683+0000 7f1bc0f61640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 0x7f1ba80a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0fea60 tx=0x7f1bac005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:28.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.686+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1bb809c0b0 con 0x7f1bbc102a70 2024-08-16T19:21:28.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.912+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bbc199460 con 0x7f1bbc102a70 2024-08-16T19:21:28.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.913+0000 7f1bb27fc640 1 -- 172.21.15.62:0/3166046989 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+12002 (secure 0 0 0) 0x7f1bbc199460 con 0x7f1bbc102a70 2024-08-16T19:21:28.914 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:21:28.914 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":56,"btime":"2024-08-16T19:20:39:638044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:20:39.561244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:21:28.914 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 56 2024-08-16T19:21:28.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.915+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 msgr2=0x7f1ba80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:28.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.915+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 0x7f1ba80a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0fea60 tx=0x7f1bac005f90 comp rx=0 tx=0).stop 2024-08-16T19:21:28.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.915+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 msgr2=0x7f1bbc198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:28.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.915+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e9f0 tx=0x7f1bb800eeb0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 shutdown_connections 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f1ba80a2d70 0x7f1ba80a5230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc10bea0 0x7f1bbc19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bbc103390 0x7f1bbc198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 --2- 172.21.15.62:0/3166046989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bbc102a70 0x7f1bbc198100 secure :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800e9f0 tx=0x7f1bb800eeb0 comp rx=0 tx=0).stop 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 >> 172.21.15.62:0/3166046989 conn(0x7f1bbc0fc4d0 msgr2=0x7f1bbc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 shutdown_connections 2024-08-16T19:21:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:28.916+0000 7f1bc39c6640 1 -- 172.21.15.62:0/3166046989 wait complete. 2024-08-16T19:21:28.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 526672 4.43 MB/sec execute 467 sec latency 43.825 ms 2024-08-16T19:21:29.641 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0' 2024-08-16T19:21:29.641 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0 2024-08-16T19:21:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:29 smithi062 ceph-mon[32993]: pgmap v343: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:29.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:29 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3166046989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:29 smithi161 ceph-mon[37576]: pgmap v343: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:29.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:29 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3166046989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:29.932 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:29.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 528930 4.45 MB/sec execute 468 sec latency 19.308 ms 2024-08-16T19:21:30.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:29 smithi155 ceph-mon[37413]: pgmap v343: 129 pgs: 129 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 56 op/s 2024-08-16T19:21:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:29 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3166046989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 -- 172.21.15.62:0/3349811142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180100700 msgr2=0x7fc180100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 --2- 172.21.15.62:0/3349811142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180100700 0x7fc180100ae0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc170009900 tx=0x7fc17002a410 comp rx=0 tx=0).stop 2024-08-16T19:21:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 -- 172.21.15.62:0/3349811142 shutdown_connections 2024-08-16T19:21:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 --2- 172.21.15.62:0/3349811142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 0x7fc180110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 --2- 172.21.15.62:0/3349811142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1801010b0 0x7fc18010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 --2- 172.21.15.62:0/3349811142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc180100700 0x7fc180100ae0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 -- 172.21.15.62:0/3349811142 >> 172.21.15.62:0/3349811142 conn(0x7fc1800fc500 msgr2=0x7fc1800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 -- 172.21.15.62:0/3349811142 shutdown_connections 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.505+0000 7fc186719640 1 -- 172.21.15.62:0/3349811142 wait complete. 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.506+0000 7fc186719640 1 Processor -- start 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.506+0000 7fc186719640 1 -- start start 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 0x7fc18019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 0x7fc18019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 0x7fc18019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc180113f90 con 0x7fc1801010b0 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc180113e10 con 0x7fc18010e220 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc186719640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc180114110 con 0x7fc180100700 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 0x7fc18019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 0x7fc18019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 0x7fc18019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53694/0 (socket says 172.21.15.62:53694) 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17f7fe640 1 -- 172.21.15.62:0/4196580219 learned_addr learned my addr 172.21.15.62:0/4196580219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc184cb6640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 0x7fc18019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 -- 172.21.15.62:0/4196580219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 msgr2=0x7fc18019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 0x7fc18019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 -- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 msgr2=0x7fc18019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 0x7fc18019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 -- 172.21.15.62:0/4196580219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18019f540 con 0x7fc180100700 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.507+0000 7fc17ffff640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 0x7fc18019a590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc170005940 tx=0x7fc1700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.508+0000 7fc17d7fa640 1 -- 172.21.15.62:0/4196580219 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17003f030 con 0x7fc180100700 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.508+0000 7fc17d7fa640 1 -- 172.21.15.62:0/4196580219 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700044c0 con 0x7fc180100700 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.508+0000 7fc17d7fa640 1 -- 172.21.15.62:0/4196580219 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170035070 con 0x7fc180100700 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.508+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18019f7d0 con 0x7fc180100700 2024-08-16T19:21:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.508+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18019fc60 con 0x7fc180100700 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.510+0000 7fc17d7fa640 1 -- 172.21.15.62:0/4196580219 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc170042030 con 0x7fc180100700 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc17d7fa640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 0x7fc1680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 0x7fc1680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc17d7fa640 1 -- 172.21.15.62:0/4196580219 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fc1700ef620 con 0x7fc180100700 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc188b17740 1 Processor -- start 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc188b17740 1 -- start start 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.511+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 0x7fc1680a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc180102500 tx=0x7fc174005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fbf3810 con 0x55ca1fc994a0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fb9ac70 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc188b17740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fbef7d0 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:36726/0 (socket says 172.21.15.62:36726) 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 -- 172.21.15.62:0/2317254417 learned_addr learned my addr 172.21.15.62:0/2317254417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc184cb6640 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 -- 172.21.15.62:0/2317254417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 msgr2=0x55ca1fc98f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc98f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 -- 172.21.15.62:0/2317254417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 msgr2=0x55ca1fc9b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.512+0000 7fc17ffff640 1 -- 172.21.15.62:0/2317254417 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1f965ba0 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc17ffff640 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1700ec9e0 tx=0x7fc1700b7bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b49870c835ed9f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc1667fc640 1 -- 172.21.15.62:0/2317254417 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700ef050 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc1667fc640 1 -- 172.21.15.62:0/2317254417 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700c43c0 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc1667fc640 1 -- 172.21.15.62:0/2317254417 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170049040 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc188b17740 1 -- 172.21.15.62:0/2317254417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 msgr2=0x55ca1fc8e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.513+0000 7fc188b17740 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc1700ec9e0 tx=0x7fc1700b7bd0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- 172.21.15.62:0/2317254417 shutdown_connections 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc98f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- 172.21.15.62:0/2317254417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc8e6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- 172.21.15.62:0/2317254417 >> 172.21.15.62:0/2317254417 conn(0x55ca1fc8a850 msgr2=0x55ca1fc8ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- 172.21.15.62:0/2317254417 shutdown_connections 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- 172.21.15.62:0/2317254417 wait complete. 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- start start 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 Processor -- start 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fc96d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fc9be30 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fbae9a0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.514+0000 7fc188b17740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ca1fc9bfb0 con 0x55ca1fc994a0 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc188b17740 10 client.0 _ll_register_callbacks cb 0x55ca1fb9cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:56688/0 (socket says 172.21.15.62:56688) 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3885003355 learned_addr learned my addr 172.21.15.62:0/3885003355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53716/0 (socket says 172.21.15.62:53716) 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc184cb6640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fc96d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3885003355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 msgr2=0x55ca1fc96d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fc96d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3885003355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 msgr2=0x55ca1fc96310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3885003355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1fbba220 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17ffff640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc17401ba10 tx=0x7fc174013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c5272e580b2c7c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc15f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:21:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.515+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17401ac20 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fc17401adc0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1700c43c0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fc174009e30 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 10 client.25565 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:30.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ca1fca79b0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1700c43c0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7fc174026070 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 10 client.25565 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 10 client.25565 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 20 client.25565 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fca8f10 0x55ca1fcab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 4 client.25565 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca1fc00f40 con 0x55ca1fca8f10 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.516+0000 7fc17ffff640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fca8f10 0x55ca1fcab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.517+0000 7fc15f7fe640 10 client.25565 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.517+0000 7fc17ffff640 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fca8f10 0x55ca1fcab380 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad58ae647cfe05f server_cookie=e29a1048d6e18907 in_seq=0 out_seq=0 2024-08-16T19:21:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc15f7fe640 1 -- 172.21.15.62:0/3885003355 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca1fc00f40 con 0x55ca1fca8f10 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc15f7fe640 10 client.25565 handle_command_reply: tid=0 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 client.25565 shutdown 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 msgr2=0x55ca1fc96850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc17401ba10 tx=0x7fc174013040 comp rx=0 tx=0).stop 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 shutdown_connections 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fca8f10 0x55ca1fcab380 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fc96d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc15f7fe640 0 client.25565 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc96850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 --2- 172.21.15.62:0/3885003355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc96310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 >> 172.21.15.62:0/3885003355 conn(0x55ca1fc8a850 msgr2=0x55ca1fc89b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.518+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 shutdown_connections 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.519+0000 7fc188b17740 1 -- 172.21.15.62:0/3885003355 wait complete. 2024-08-16T19:21:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.519+0000 7fc188b17740 20 client.25565 trim_cache size 0 max 16384 2024-08-16T19:21:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 Processor -- start 2024-08-16T19:21:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 -- start start 2024-08-16T19:21:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fcb0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801b1150 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801b2d50 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc188b17740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801b1820 con 0x55ca1fc994a0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.536+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53728/0 (socket says 172.21.15.62:53728) 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3160884422 learned_addr learned my addr 172.21.15.62:0/3160884422 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc184cb6640 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fcb0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3160884422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 msgr2=0x55ca1fcb0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fcb0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3160884422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 msgr2=0x55ca1fc85ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc85ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 -- 172.21.15.62:0/3160884422 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1fc952c0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc174027980 tx=0x7fc1740091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28046845d2a260ef server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc165ffb640 1 -- 172.21.15.62:0/3160884422 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17403a020 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc165ffb640 1 -- 172.21.15.62:0/3160884422 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174011d20 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.537+0000 7fc165ffb640 1 -- 172.21.15.62:0/3160884422 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740054a0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- 172.21.15.62:0/3160884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 msgr2=0x55ca1fc89a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc174027980 tx=0x7fc1740091e0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- 172.21.15.62:0/3160884422 shutdown_connections 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc994a0 0x55ca1fcb0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc89a20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 --2- 172.21.15.62:0/3160884422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fc85ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- 172.21.15.62:0/3160884422 >> 172.21.15.62:0/3160884422 conn(0x55ca1fc8a850 msgr2=0x55ca1fc89310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- 172.21.15.62:0/3160884422 shutdown_connections 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- 172.21.15.62:0/3160884422 wait complete. 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.538+0000 7fc188b17740 1 -- start start 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 Processor -- start 2024-08-16T19:21:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801b25c0 con 0x55ca1fc994a0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801b1ef0 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801af0f0 con 0x55ca1fc8ebe0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc188b17740 10 client.0 _ll_register_callbacks cb 0x7fc174009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc184cb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc17ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:56710/0 (socket says 172.21.15.62:56710) 2024-08-16T19:21:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc17ffff640 1 -- 172.21.15.62:0/180710945 learned_addr learned my addr 172.21.15.62:0/180710945 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.539+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 -- 172.21.15.62:0/180710945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8ebe0 msgr2=0x55ca1fc9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 -- 172.21.15.62:0/180710945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 msgr2=0x55ca1fc9d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 -- 172.21.15.62:0/180710945 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1fd12c60 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc184cb6640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc17ffff640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b7b10 tx=0x7fc1700bb980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316640a79aa51ce3 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc15f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700ef040 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fc1700ecd20 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc174011d20 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fc170049040 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc188b17740 10 client.25571 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ca1fd13e00 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.540+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc174011d20 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7fc1700b7e70 con 0x55ca1fc8e2c0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 10 client.25571 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 10 client.25571 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 20 client.25571 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fd14310 0x55ca1fd16730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 4 client.25571 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}] 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}) -- 0x55ca1fbfb290 con 0x55ca1fd14310 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fd14310 0x55ca1fd16730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc17f7fe640 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fd14310 0x55ca1fd16730 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed45017d65b96c4b server_cookie=aa86f096d8218cc2 in_seq=0 out_seq=0 2024-08-16T19:21:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.541+0000 7fc15f7fe640 10 client.25571 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc15f7fe640 1 -- 172.21.15.62:0/180710945 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ca1fbfb290 con 0x55ca1fd14310 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc15f7fe640 10 client.25571 handle_command_reply: tid=0 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 client.25571 shutdown 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 msgr2=0x55ca1fca1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1700b7b10 tx=0x7fc1700bb980 comp rx=0 tx=0).stop 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 shutdown_connections 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55ca1fd14310 0x55ca1fd16730 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ca1fc994a0 0x55ca1fc9d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc15f7fe640 0 client.25571 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ca1fc8ebe0 0x55ca1fc9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 --2- 172.21.15.62:0/180710945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca1fc8e2c0 0x55ca1fca1a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 >> 172.21.15.62:0/180710945 conn(0x55ca1fc8a850 msgr2=0x55ca1fc89b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 shutdown_connections 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 1 -- 172.21.15.62:0/180710945 wait complete. 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.611+0000 7fc188b17740 20 client.25571 trim_cache size 0 max 16384 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:21:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 56, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 42, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "aacd0a0f": { 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 42, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.9, 2024-08-16T19:21:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:30.615 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 msgr2=0x7fc1680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 0x7fc1680a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc180102500 tx=0x7fc174005f90 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 msgr2=0x7fc18019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 0x7fc18019a590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc170005940 tx=0x7fc1700040c0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 shutdown_connections 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc1680a2d70 0x7fc1680a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc18010e220 0x7fc18019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1801010b0 0x7fc18019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 --2- 172.21.15.62:0/4196580219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc180100700 0x7fc18019a590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 >> 172.21.15.62:0/4196580219 conn(0x7fc1800fc500 msgr2=0x7fc1800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:30.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 shutdown_connections 2024-08-16T19:21:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:30.612+0000 7fc186719640 1 -- 172.21.15.62:0/4196580219 wait complete. 2024-08-16T19:21:30.985 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.45 MB/sec execute 469 sec latency 443.539 ms 2024-08-16T19:21:31.354 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'aacd0a0f', quiesce took 3.5 seconds. Will release after: 24.45 2024-08-16T19:21:31.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:31 smithi161 ceph-mon[37576]: pgmap v344: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-08-16T19:21:31.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.44 MB/sec execute 470 sec latency 1443.619 ms 2024-08-16T19:21:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:31 smithi062 ceph-mon[32993]: pgmap v344: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-08-16T19:21:32.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:31 smithi155 ceph-mon[37413]: pgmap v344: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-08-16T19:21:32.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.43 MB/sec execute 471 sec latency 2443.706 ms 2024-08-16T19:21:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:33 smithi161 ceph-mon[37576]: pgmap v345: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-08-16T19:21:33.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.42 MB/sec execute 472 sec latency 3443.827 ms 2024-08-16T19:21:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:33 smithi062 ceph-mon[32993]: pgmap v345: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-08-16T19:21:34.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:33 smithi155 ceph-mon[37413]: pgmap v345: 129 pgs: 129 active+clean; 420 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-08-16T19:21:34.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.41 MB/sec execute 473 sec latency 4443.977 ms 2024-08-16T19:21:35.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:35 smithi161 ceph-mon[37576]: pgmap v346: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-08-16T19:21:35.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.40 MB/sec execute 474 sec latency 5444.100 ms 2024-08-16T19:21:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:35 smithi062 ceph-mon[32993]: pgmap v346: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-08-16T19:21:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:35 smithi155 ceph-mon[37413]: pgmap v346: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-08-16T19:21:36.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.39 MB/sec execute 475 sec latency 6444.241 ms 2024-08-16T19:21:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:37 smithi161 ceph-mon[37576]: pgmap v347: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:37.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.38 MB/sec execute 476 sec latency 7444.343 ms 2024-08-16T19:21:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:37 smithi062 ceph-mon[32993]: pgmap v347: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:37 smithi155 ceph-mon[37413]: pgmap v347: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:38.986 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.37 MB/sec execute 477 sec latency 8444.445 ms 2024-08-16T19:21:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:39 smithi161 ceph-mon[37576]: pgmap v348: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:39.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.36 MB/sec execute 478 sec latency 9444.524 ms 2024-08-16T19:21:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:39 smithi062 ceph-mon[32993]: pgmap v348: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:39 smithi155 ceph-mon[37413]: pgmap v348: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-08-16T19:21:40.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.36 MB/sec execute 479 sec latency 10444.636 ms 2024-08-16T19:21:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:41 smithi161 ceph-mon[37576]: pgmap v349: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-08-16T19:21:41.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.35 MB/sec execute 480 sec latency 11444.757 ms 2024-08-16T19:21:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:41 smithi062 ceph-mon[32993]: pgmap v349: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-08-16T19:21:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:41 smithi155 ceph-mon[37413]: pgmap v349: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-08-16T19:21:42.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.34 MB/sec execute 481 sec latency 12444.855 ms 2024-08-16T19:21:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:43 smithi161 ceph-mon[37576]: pgmap v350: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 614 KiB/s wr, 57 op/s 2024-08-16T19:21:43.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.33 MB/sec execute 482 sec latency 13444.908 ms 2024-08-16T19:21:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:43 smithi062 ceph-mon[32993]: pgmap v350: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 614 KiB/s wr, 57 op/s 2024-08-16T19:21:44.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:43 smithi155 ceph-mon[37413]: pgmap v350: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 614 KiB/s wr, 57 op/s 2024-08-16T19:21:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:21:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:21:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:21:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:21:44.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.32 MB/sec execute 483 sec latency 14445.034 ms 2024-08-16T19:21:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:21:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:21:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:21:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:21:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:21:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:21:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:21:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:21:45.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.31 MB/sec execute 484 sec latency 15445.122 ms 2024-08-16T19:21:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:46 smithi161 ceph-mon[37576]: pgmap v351: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 627 KiB/s wr, 63 op/s 2024-08-16T19:21:46.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:46 smithi062 ceph-mon[32993]: pgmap v351: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 627 KiB/s wr, 63 op/s 2024-08-16T19:21:46.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:46 smithi155 ceph-mon[37413]: pgmap v351: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 627 KiB/s wr, 63 op/s 2024-08-16T19:21:46.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.30 MB/sec execute 485 sec latency 16445.226 ms 2024-08-16T19:21:47.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.29 MB/sec execute 486 sec latency 17445.299 ms 2024-08-16T19:21:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:48 smithi161 ceph-mon[37576]: pgmap v352: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:48.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:48 smithi062 ceph-mon[32993]: pgmap v352: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:48.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:48 smithi155 ceph-mon[37413]: pgmap v352: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:48.987 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.28 MB/sec execute 487 sec latency 18445.397 ms 2024-08-16T19:21:49.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.28 MB/sec execute 488 sec latency 19445.465 ms 2024-08-16T19:21:50.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:50 smithi161 ceph-mon[37576]: pgmap v353: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:50.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:50 smithi062 ceph-mon[32993]: pgmap v353: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:50.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:50 smithi155 ceph-mon[37413]: pgmap v353: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-08-16T19:21:50.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.27 MB/sec execute 489 sec latency 20445.548 ms 2024-08-16T19:21:51.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.26 MB/sec execute 490 sec latency 21445.643 ms 2024-08-16T19:21:52.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:52 smithi161 ceph-mon[37576]: pgmap v354: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 608 KiB/s wr, 33 op/s 2024-08-16T19:21:52.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:52 smithi062 ceph-mon[32993]: pgmap v354: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 608 KiB/s wr, 33 op/s 2024-08-16T19:21:52.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:52 smithi155 ceph-mon[37413]: pgmap v354: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 608 KiB/s wr, 33 op/s 2024-08-16T19:21:52.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.25 MB/sec execute 491 sec latency 22445.767 ms 2024-08-16T19:21:53.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.24 MB/sec execute 492 sec latency 23445.842 ms 2024-08-16T19:21:54.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:54 smithi161 ceph-mon[37576]: pgmap v355: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 571 KiB/s wr, 22 op/s 2024-08-16T19:21:54.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:54 smithi062 ceph-mon[32993]: pgmap v355: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 571 KiB/s wr, 22 op/s 2024-08-16T19:21:54.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:54 smithi155 ceph-mon[37413]: pgmap v355: 129 pgs: 129 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 571 KiB/s wr, 22 op/s 2024-08-16T19:21:54.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.23 MB/sec execute 493 sec latency 24445.978 ms 2024-08-16T19:21:55.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:21:55.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.22 MB/sec execute 494 sec latency 25446.025 ms 2024-08-16T19:21:56.084 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:56 smithi161 ceph-mon[37576]: pgmap v356: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 618 KiB/s wr, 34 op/s 2024-08-16T19:21:56.395 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:56 smithi062 ceph-mon[32993]: pgmap v356: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 618 KiB/s wr, 34 op/s 2024-08-16T19:21:56.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:56 smithi155 ceph-mon[37413]: pgmap v356: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 618 KiB/s wr, 34 op/s 2024-08-16T19:21:56.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.680+0000 7fce1d782640 1 -- 172.21.15.62:0/2745068212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce18102c20 msgr2=0x7fce1810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:56.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.680+0000 7fce1d782640 1 --2- 172.21.15.62:0/2745068212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce18102c20 0x7fce1810d770 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fce10009ca0 tx=0x7fce1002a950 comp rx=0 tx=0).stop 2024-08-16T19:21:56.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.680+0000 7fce1d782640 1 -- 172.21.15.62:0/2745068212 shutdown_connections 2024-08-16T19:21:56.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 --2- 172.21.15.62:0/2745068212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce18102c20 0x7fce1810d770 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 --2- 172.21.15.62:0/2745068212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce181022a0 0x7fce181026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 --2- 172.21.15.62:0/2745068212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18100b50 0x7fce18100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 -- 172.21.15.62:0/2745068212 >> 172.21.15.62:0/2745068212 conn(0x7fce180fc500 msgr2=0x7fce180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 -- 172.21.15.62:0/2745068212 shutdown_connections 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 -- 172.21.15.62:0/2745068212 wait complete. 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.681+0000 7fce1d782640 1 Processor -- start 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 -- start start 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18100b50 0x7fce1819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18102c20 0x7fce1819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce18110cd0 con 0x7fce181022a0 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fce18110b50 con 0x7fce18100b50 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce1d782640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce18110e50 con 0x7fce18102c20 2024-08-16T19:21:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52746/0 (socket says 172.21.15.62:52746) 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 -- 172.21.15.62:0/906316290 learned_addr learned my addr 172.21.15.62:0/906316290 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce16ffd640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18100b50 0x7fce1819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce177fe640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18102c20 0x7fce1819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 -- 172.21.15.62:0/906316290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18102c20 msgr2=0x7fce1819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18102c20 0x7fce1819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 -- 172.21.15.62:0/906316290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18100b50 msgr2=0x7fce1819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18100b50 0x7fce1819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.682+0000 7fce167fc640 1 -- 172.21.15.62:0/906316290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1819f300 con 0x7fce181022a0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fce167fc640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d950 tx=0x7fce0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c017070 con 0x7fce181022a0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1819f5f0 con 0x7fce181022a0 2024-08-16T19:21:56.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce0c009c30 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00f6b0 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.683+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181a6ed0 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.684+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18100f30 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.686+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce0c0109d0 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.687+0000 7fcdfffff640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.687+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fce0c0cf3d0 con 0x7fce181022a0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.687+0000 7fce16ffd640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.688+0000 7fce16ffd640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 0x7fce000a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce0800bbe0 tx=0x7fce0800b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:56.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.689+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fce0c009de0 con 0x7fce181022a0 2024-08-16T19:21:56.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.901+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce1819b6b0 con 0x7fce181022a0 2024-08-16T19:21:56.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.902+0000 7fcdfffff640 1 -- 172.21.15.62:0/906316290 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+12002 (secure 0 0 0) 0x7fce1819b6b0 con 0x7fce181022a0 2024-08-16T19:21:56.902 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:21:56.903 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":56,"btime":"2024-08-16T19:20:39:638044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:20:39.561244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:21:56.903 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 56 2024-08-16T19:21:56.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 msgr2=0x7fce000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:56.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 0x7fce000a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fce0800bbe0 tx=0x7fce0800b390 comp rx=0 tx=0).stop 2024-08-16T19:21:56.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 msgr2=0x7fce1819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d950 tx=0x7fce0c00de10 comp rx=0 tx=0).stop 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 shutdown_connections 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fce000a2cf0 0x7fce000a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce18102c20 0x7fce1819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce181022a0 0x7fce1819a890 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 --2- 172.21.15.62:0/906316290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18100b50 0x7fce1819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 >> 172.21.15.62:0/906316290 conn(0x7fce180fc500 msgr2=0x7fce1806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 shutdown_connections 2024-08-16T19:21:56.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:56.904+0000 7fce1d782640 1 -- 172.21.15.62:0/906316290 wait complete. 2024-08-16T19:21:56.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.21 MB/sec execute 495 sec latency 26446.073 ms 2024-08-16T19:21:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/906316290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:57.403 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/906316290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:57.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/906316290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:21:57.820 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id aacd0a0f --release --await' 2024-08-16T19:21:57.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id aacd0a0f --release --await 2024-08-16T19:21:57.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 529728 4.21 MB/sec execute 496 sec latency 27446.108 ms 2024-08-16T19:21:58.088 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:21:58.379 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:21:58 smithi062 ceph-mon[32993]: pgmap v357: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:21:58.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:21:58 smithi161 ceph-mon[37576]: pgmap v357: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:21:58.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 -- 172.21.15.62:0/393049847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc10bea0 msgr2=0x7f3efc10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 --2- 172.21.15.62:0/393049847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc10bea0 0x7f3efc10e340 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009ca0 tx=0x7f3ef402a950 comp rx=0 tx=0).stop 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 -- 172.21.15.62:0/393049847 shutdown_connections 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 --2- 172.21.15.62:0/393049847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc10bea0 0x7f3efc10e340 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 --2- 172.21.15.62:0/393049847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc103390 0x7f3efc10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 --2- 172.21.15.62:0/393049847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 0x7f3efc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.570+0000 7f3f00a37640 1 -- 172.21.15.62:0/393049847 >> 172.21.15.62:0/393049847 conn(0x7f3efc0fc4d0 msgr2=0x7f3efc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.571+0000 7f3f00a37640 1 -- 172.21.15.62:0/393049847 shutdown_connections 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.571+0000 7f3f00a37640 1 -- 172.21.15.62:0/393049847 wait complete. 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.571+0000 7f3f00a37640 1 Processor -- start 2024-08-16T19:21:58.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.571+0000 7f3f00a37640 1 -- start start 2024-08-16T19:21:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 0x7f3efc1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc10bea0 0x7f3efc101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111ca0 con 0x7f3efc103390 2024-08-16T19:21:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111b20 con 0x7f3efc10bea0 2024-08-16T19:21:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3f00a37640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc111e20 con 0x7f3efc102a70 2024-08-16T19:21:58.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52768/0 (socket says 172.21.15.62:52768) 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 0x7f3efc1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 -- 172.21.15.62:0/339802520 learned_addr learned my addr 172.21.15.62:0/339802520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3efad9d640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc10bea0 0x7f3efc101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 -- 172.21.15.62:0/339802520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 msgr2=0x7f3efc1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 0x7f3efc1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 -- 172.21.15.62:0/339802520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc10bea0 msgr2=0x7f3efc101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc10bea0 0x7f3efc101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 -- 172.21.15.62:0/339802520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a65e0 con 0x7f3efc103390 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.572+0000 7f3ef9d9b640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c940 tx=0x7f3ef000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.573+0000 7f3eeb7fe640 1 -- 172.21.15.62:0/339802520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004220 con 0x7f3efc103390 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.573+0000 7f3eeb7fe640 1 -- 172.21.15.62:0/339802520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef00043c0 con 0x7f3efc103390 2024-08-16T19:21:58.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.573+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a68d0 con 0x7f3efc103390 2024-08-16T19:21:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.573+0000 7f3eeb7fe640 1 -- 172.21.15.62:0/339802520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004a00 con 0x7f3efc103390 2024-08-16T19:21:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.573+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a6e60 con 0x7f3efc103390 2024-08-16T19:21:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.575+0000 7f3eeb7fe640 1 -- 172.21.15.62:0/339802520 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3ef00079b0 con 0x7f3efc103390 2024-08-16T19:21:58.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.575+0000 7f3f02e35740 1 Processor -- start 2024-08-16T19:21:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.575+0000 7f3f02e35740 1 -- start start 2024-08-16T19:21:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3eeb7fe640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 0x7f3ee40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e7ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da07dd8c60 con 0x55da07e732b0 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da07d70b90 con 0x55da07e7db70 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3f02e35740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55da07dd4c20 con 0x55da07e72990 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52780/0 (socket says 172.21.15.62:52780) 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 -- 172.21.15.62:0/1151714822 learned_addr learned my addr 172.21.15.62:0/1151714822 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:58.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3ef9d9b640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 0x7f3ee40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efad9d640 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efad9d640 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e7ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3eeb7fe640 1 -- 172.21.15.62:0/339802520 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f3ef00cf640 con 0x7f3efc103390 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 -- 172.21.15.62:0/1151714822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 msgr2=0x55da07e72d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e72d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 -- 172.21.15.62:0/1151714822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 msgr2=0x55da07e7ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e7ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.576+0000 7f3efa59c640 1 -- 172.21.15.62:0/1151714822 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da07ae8580 con 0x55da07e732b0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.577+0000 7f3efa59c640 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ac10 tx=0x7f3eec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ebdf5d377cd38b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.577+0000 7f3ef9d9b640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 0x7f3ee40a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ef009b7b0 tx=0x7f3ef00c6300 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.577+0000 7f3ee8ff9640 1 -- 172.21.15.62:0/1151714822 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec01d070 con 0x55da07e732b0 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.577+0000 7f3ee8ff9640 1 -- 172.21.15.62:0/1151714822 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eec018b20 con 0x55da07e732b0 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.577+0000 7f3ee8ff9640 1 -- 172.21.15.62:0/1151714822 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec010af0 con 0x55da07e732b0 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- 172.21.15.62:0/1151714822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 msgr2=0x55da07e7d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ac10 tx=0x7f3eec00f790 comp rx=0 tx=0).stop 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- 172.21.15.62:0/1151714822 shutdown_connections 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e7ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e7d630 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- 172.21.15.62:0/1151714822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e72d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- 172.21.15.62:0/1151714822 >> 172.21.15.62:0/1151714822 conn(0x55da07e6ef20 msgr2=0x55da07e6f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- 172.21.15.62:0/1151714822 shutdown_connections 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- 172.21.15.62:0/1151714822 wait complete. 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- start start 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 Processor -- start 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e732b0 0x55da07e7afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e7b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55da07e80740 con 0x55da07e72990 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55da07d767c0 con 0x55da07e732b0 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55da07e808c0 con 0x55da07e7db70 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3f02e35740 10 client.0 _ll_register_callbacks cb 0x55da07d7ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.578+0000 7f3efad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52784/0 (socket says 172.21.15.62:52784) 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 -- 172.21.15.62:0/3431790806 learned_addr learned my addr 172.21.15.62:0/3431790806 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efa59c640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e732b0 0x55da07e7afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 -- 172.21.15.62:0/3431790806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 msgr2=0x55da07e7b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e7b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 -- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e732b0 msgr2=0x55da07e7afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e732b0 0x55da07e7afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 -- 172.21.15.62:0/3431790806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da07bb2960 con 0x55da07e72990 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3efad9d640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009ca0 tx=0x7f3ef402ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97d622a077dd11 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:21:58.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4034040 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f3ef4003b80 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eec018b20 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3ef4003d80 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3f02e35740 10 client.16044 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.579+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55da07e8c780 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eec018b20 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f3ef4048020 con 0x55da07e72990 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 10 client.16044 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:58.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 10 client.16044 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 20 client.16044 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07e8dc90 0x55da07e900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 4 client.16044 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da07de61d0 con 0x55da07e8dc90 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.580+0000 7f3efa59c640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07e8dc90 0x55da07e900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3ee1ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3efa59c640 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07e8dc90 0x55da07e900d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f39f2a16e1ee24 server_cookie=15b0ff09d7ac2ecb in_seq=0 out_seq=0 2024-08-16T19:21:58.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/3431790806 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da07de61d0 con 0x55da07e8dc90 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3ee1ffb640 10 client.16044 handle_command_reply: tid=0 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3f02e35740 1 client.16044 shutdown 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 msgr2=0x55da07e7aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.581+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009ca0 tx=0x7f3ef402ff30 comp rx=0 tx=0).stop 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 shutdown_connections 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07e8dc90 0x55da07e900d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e7b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e732b0 0x55da07e7afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 --2- 172.21.15.62:0/3431790806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e72990 0x55da07e7aa60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 >> 172.21.15.62:0/3431790806 conn(0x55da07e6ef20 msgr2=0x55da07e74c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3ee1ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 shutdown_connections 2024-08-16T19:21:58.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 1 -- 172.21.15.62:0/3431790806 wait complete. 2024-08-16T19:21:58.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.582+0000 7f3f02e35740 20 client.16044 trim_cache size 0 max 16384 2024-08-16T19:21:58.594 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:21:58 smithi155 ceph-mon[37413]: pgmap v357: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.599+0000 7f3f02e35740 1 Processor -- start 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.599+0000 7f3f02e35740 1 -- start start 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e69af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e95f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b1160 con 0x55da07e732b0 2024-08-16T19:21:58.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b2d60 con 0x55da07e7db70 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3f02e35740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b1830 con 0x55da07e72990 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52786/0 (socket says 172.21.15.62:52786) 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 -- 172.21.15.62:0/2399876439 learned_addr learned my addr 172.21.15.62:0/2399876439 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efad9d640 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e69af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efad9d640 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e95f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 -- 172.21.15.62:0/2399876439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 msgr2=0x55da07e69af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e69af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 -- 172.21.15.62:0/2399876439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 msgr2=0x55da07e95f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e95f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 -- 172.21.15.62:0/2399876439 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da07e79a50 con 0x55da07e732b0 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.600+0000 7f3efa59c640 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ac10 tx=0x7f3eec00a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f5b14a92bfbdce server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3ee3fff640 1 -- 172.21.15.62:0/2399876439 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec032070 con 0x55da07e732b0 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3ee3fff640 1 -- 172.21.15.62:0/2399876439 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eec01e040 con 0x55da07e732b0 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3ee3fff640 1 -- 172.21.15.62:0/2399876439 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec01d040 con 0x55da07e732b0 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 -- 172.21.15.62:0/2399876439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 msgr2=0x55da07e959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00ac10 tx=0x7f3eec00a700 comp rx=0 tx=0).stop 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 -- 172.21.15.62:0/2399876439 shutdown_connections 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e7db70 0x55da07e95f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e959e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 --2- 172.21.15.62:0/2399876439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e72990 0x55da07e69af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 -- 172.21.15.62:0/2399876439 >> 172.21.15.62:0/2399876439 conn(0x55da07e6ef20 msgr2=0x55da07e74c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 -- 172.21.15.62:0/2399876439 shutdown_connections 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.601+0000 7f3f02e35740 1 -- 172.21.15.62:0/2399876439 wait complete. 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 -- start start 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 Processor -- start 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e72990 0x55da07e86170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e83020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b25d0 con 0x55da07e732b0 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1b1f00 con 0x55da07e72990 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc1af100 con 0x55da07e7db70 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3f02e35740 10 client.0 _ll_register_callbacks cb 0x7f3ef402f6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3efa59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52800/0 (socket says 172.21.15.62:52800) 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3efa59c640 1 -- 172.21.15.62:0/730980885 learned_addr learned my addr 172.21.15.62:0/730980885 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3efad9d640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e72990 0x55da07e86170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.602+0000 7f3efad9d640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e83020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 -- 172.21.15.62:0/730980885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 msgr2=0x55da07e83020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e83020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 -- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e72990 msgr2=0x55da07e86170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e72990 0x55da07e86170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 -- 172.21.15.62:0/730980885 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da07e836c0 con 0x55da07e732b0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efad9d640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e83020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3efa59c640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a6d0 tx=0x7f3eec00ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cafdc8588cb47a server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec032070 con 0x55da07e732b0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f3eec0187e0 con 0x55da07e732b0 2024-08-16T19:21:58.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eec01e040 con 0x55da07e732b0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3eec01d040 con 0x55da07e732b0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3f02e35740 10 client.16050 fetch_fsmap learned FSMap version 56 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55da07ef9890 con 0x55da07e732b0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eec01e040 con 0x55da07e732b0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.603+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 56) ==== 8402+0+0 (secure 0 0 0) 0x7f3eec004a70 con 0x55da07e732b0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 10 client.16050 fetch_fsmap finished waiting for FSMap version 56 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 10 client.16050 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 20 client.16050 populate_metadata read hostname 'smithi062' 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07ef9e30 0x55da07efc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 4 client.16050 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "aacd0a0f", "release": true, "await": true}] 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "aacd0a0f", "release": true, "await": true}) -- 0x55da07de06e0 con 0x55da07ef9e30 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3efad9d640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07ef9e30 0x55da07efc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:21:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3efad9d640 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07ef9e30 0x55da07efc250 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd824f46763fd40d server_cookie=9f489d80b398f938 in_seq=0 out_seq=0 2024-08-16T19:21:58.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.604+0000 7f3ee1ffb640 10 client.16050 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3ee1ffb640 1 -- 172.21.15.62:0/730980885 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55da07de06e0 con 0x55da07ef9e30 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3ee1ffb640 10 client.16050 handle_command_reply: tid=0 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 client.16050 shutdown 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 msgr2=0x55da07e82ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a6d0 tx=0x7f3eec00ab90 comp rx=0 tx=0).stop 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 shutdown_connections 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55da07ef9e30 0x55da07efc250 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55da07e7db70 0x55da07e83020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55da07e732b0 0x55da07e82ae0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 --2- 172.21.15.62:0/730980885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55da07e72990 0x55da07e86170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 >> 172.21.15.62:0/730980885 conn(0x55da07e6ef20 msgr2=0x55da07e6d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.613+0000 7f3ee1ffb640 0 client.16050 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 shutdown_connections 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f02e35740 1 -- 172.21.15.62:0/730980885 wait complete. 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f02e35740 20 client.16050 trim_cache size 0 max 16384 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 56, 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:21:58.616 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 44, 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "aacd0a0f": { 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 44, 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:21:58.617 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 147.9, 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:21:58.618 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 msgr2=0x7f3ee40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 0x7f3ee40a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ef009b7b0 tx=0x7f3ef00c6300 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 msgr2=0x7f3efc100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c940 tx=0x7f3ef000ce00 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 shutdown_connections 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3ee40a2d70 0x7f3ee40a5230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3efc10bea0 0x7f3efc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3efc103390 0x7f3efc100c10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 --2- 172.21.15.62:0/339802520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3efc102a70 0x7f3efc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 >> 172.21.15.62:0/339802520 conn(0x7f3efc0fc4d0 msgr2=0x7f3efc0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 shutdown_connections 2024-08-16T19:21:58.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:21:58.614+0000 7f3f00a37640 1 -- 172.21.15.62:0/339802520 wait complete. 2024-08-16T19:21:58.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 530219 4.20 MB/sec execute 497 sec latency 28073.189 ms 2024-08-16T19:21:59.525 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'aacd0a0f', total seconds elapsed: 31.6 2024-08-16T19:21:59.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 532356 4.21 MB/sec execute 498 sec latency 16.093 ms 2024-08-16T19:22:00.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:00 smithi161 ceph-mon[37576]: pgmap v358: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:22:00.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:00 smithi062 ceph-mon[32993]: pgmap v358: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:22:00.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:00 smithi155 ceph-mon[37413]: pgmap v358: 129 pgs: 129 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 605 KiB/s wr, 28 op/s 2024-08-16T19:22:00.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 533801 4.22 MB/sec execute 499 sec latency 32.435 ms 2024-08-16T19:22:01.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 534359 4.22 MB/sec execute 500 sec latency 55.097 ms 2024-08-16T19:22:02.387 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:02 smithi161 ceph-mon[37576]: pgmap v359: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 784 KiB/s wr, 49 op/s 2024-08-16T19:22:02.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:02 smithi062 ceph-mon[32993]: pgmap v359: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 784 KiB/s wr, 49 op/s 2024-08-16T19:22:02.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:02 smithi155 ceph-mon[37413]: pgmap v359: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 784 KiB/s wr, 49 op/s 2024-08-16T19:22:02.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 536439 4.24 MB/sec execute 501 sec latency 18.917 ms 2024-08-16T19:22:03.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 537600 4.24 MB/sec execute 502 sec latency 18.577 ms 2024-08-16T19:22:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:04 smithi161 ceph-mon[37576]: pgmap v360: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 226 KiB/s wr, 33 op/s 2024-08-16T19:22:04.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:04 smithi062 ceph-mon[32993]: pgmap v360: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 226 KiB/s wr, 33 op/s 2024-08-16T19:22:04.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:04 smithi155 ceph-mon[37413]: pgmap v360: 129 pgs: 129 active+clean; 428 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 226 KiB/s wr, 33 op/s 2024-08-16T19:22:04.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 539442 4.25 MB/sec execute 503 sec latency 17.103 ms 2024-08-16T19:22:05.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 540734 4.25 MB/sec execute 504 sec latency 36.366 ms 2024-08-16T19:22:06.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:06 smithi062 ceph-mon[32993]: pgmap v361: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-16T19:22:06.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:06 smithi062 ceph-mon[32993]: osdmap e104: 12 total, 12 up, 12 in 2024-08-16T19:22:06.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:06 smithi155 ceph-mon[37413]: pgmap v361: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-16T19:22:06.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:06 smithi155 ceph-mon[37413]: osdmap e104: 12 total, 12 up, 12 in 2024-08-16T19:22:06.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:06 smithi161 ceph-mon[37576]: pgmap v361: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-16T19:22:06.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:06 smithi161 ceph-mon[37576]: osdmap e104: 12 total, 12 up, 12 in 2024-08-16T19:22:06.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 542421 4.26 MB/sec execute 505 sec latency 20.248 ms 2024-08-16T19:22:07.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 543993 4.27 MB/sec execute 506 sec latency 15.619 ms 2024-08-16T19:22:08.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:08 smithi062 ceph-mon[32993]: pgmap v363: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:08.583 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:08 smithi155 ceph-mon[37413]: pgmap v363: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:08.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:08 smithi161 ceph-mon[37576]: pgmap v363: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:08.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 545956 4.28 MB/sec execute 507 sec latency 20.617 ms 2024-08-16T19:22:09.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 547553 4.30 MB/sec execute 508 sec latency 12.927 ms 2024-08-16T19:22:10.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:10 smithi062 ceph-mon[32993]: pgmap v364: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:10.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:10 smithi155 ceph-mon[37413]: pgmap v364: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:10.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:10 smithi161 ceph-mon[37576]: pgmap v364: 129 pgs: 129 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-08-16T19:22:10.988 INFO:tasks.workunit.client.0.smithi062.stdout: 1 549049 4.30 MB/sec execute 509 sec latency 23.438 ms 2024-08-16T19:22:11.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 551007 4.32 MB/sec execute 510 sec latency 13.825 ms 2024-08-16T19:22:12.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:12 smithi062 ceph-mon[32993]: pgmap v365: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-08-16T19:22:12.498 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:12 smithi062 ceph-mon[32993]: osdmap e105: 12 total, 12 up, 12 in 2024-08-16T19:22:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:12 smithi155 ceph-mon[37413]: pgmap v365: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-08-16T19:22:12.580 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:12 smithi155 ceph-mon[37413]: osdmap e105: 12 total, 12 up, 12 in 2024-08-16T19:22:12.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:12 smithi161 ceph-mon[37576]: pgmap v365: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 266 op/s 2024-08-16T19:22:12.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:12 smithi161 ceph-mon[37576]: osdmap e105: 12 total, 12 up, 12 in 2024-08-16T19:22:12.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 552281 4.32 MB/sec execute 511 sec latency 17.223 ms 2024-08-16T19:22:13.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 554442 4.34 MB/sec execute 512 sec latency 46.497 ms 2024-08-16T19:22:14.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:14 smithi062 ceph-mon[32993]: pgmap v367: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-08-16T19:22:14.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:14 smithi155 ceph-mon[37413]: pgmap v367: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-08-16T19:22:14.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:14 smithi161 ceph-mon[37576]: pgmap v367: 129 pgs: 129 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-08-16T19:22:14.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 555769 4.34 MB/sec execute 513 sec latency 19.063 ms 2024-08-16T19:22:15.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 557992 4.36 MB/sec execute 514 sec latency 17.671 ms 2024-08-16T19:22:16.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:16 smithi161 ceph-mon[37576]: pgmap v368: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-08-16T19:22:16.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:16 smithi062 ceph-mon[32993]: pgmap v368: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-08-16T19:22:16.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:16 smithi155 ceph-mon[37413]: pgmap v368: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-08-16T19:22:16.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 559214 4.36 MB/sec execute 515 sec latency 15.048 ms 2024-08-16T19:22:17.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:17 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:17.582 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:17 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:17.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:17 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:17.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 561472 4.38 MB/sec execute 516 sec latency 22.647 ms 2024-08-16T19:22:18.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:18 smithi062 ceph-mon[32993]: pgmap v369: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:18.579 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:18 smithi155 ceph-mon[37413]: pgmap v369: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:18.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:18 smithi161 ceph-mon[37576]: pgmap v369: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:18.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 562656 4.38 MB/sec execute 517 sec latency 47.146 ms 2024-08-16T19:22:19.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 564935 4.39 MB/sec execute 518 sec latency 16.886 ms 2024-08-16T19:22:20.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:20 smithi062 ceph-mon[32993]: pgmap v370: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:20.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:20 smithi155 ceph-mon[37413]: pgmap v370: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:20.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:20 smithi161 ceph-mon[37576]: pgmap v370: 129 pgs: 129 active+clean; 455 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-08-16T19:22:20.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 566204 4.40 MB/sec execute 519 sec latency 17.635 ms 2024-08-16T19:22:21.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 568306 4.41 MB/sec execute 520 sec latency 16.515 ms 2024-08-16T19:22:22.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:22 smithi062 ceph-mon[32993]: pgmap v371: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-08-16T19:22:22.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:22 smithi155 ceph-mon[37413]: pgmap v371: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-08-16T19:22:22.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:22 smithi161 ceph-mon[37576]: pgmap v371: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-08-16T19:22:22.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 569514 4.41 MB/sec execute 521 sec latency 17.437 ms 2024-08-16T19:22:23.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:23 smithi062 ceph-mon[32993]: pgmap v372: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-08-16T19:22:23.578 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:23 smithi155 ceph-mon[37413]: pgmap v372: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-08-16T19:22:23.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:23 smithi161 ceph-mon[37576]: pgmap v372: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 226 op/s 2024-08-16T19:22:23.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 571340 4.42 MB/sec execute 522 sec latency 20.917 ms 2024-08-16T19:22:24.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 572700 4.43 MB/sec execute 523 sec latency 16.431 ms 2024-08-16T19:22:25.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:25 smithi155 ceph-mon[37413]: pgmap v373: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2024-08-16T19:22:25.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:25 smithi161 ceph-mon[37576]: pgmap v373: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2024-08-16T19:22:25.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 574891 4.44 MB/sec execute 524 sec latency 15.617 ms 2024-08-16T19:22:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:25 smithi062 ceph-mon[32993]: pgmap v373: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 304 op/s 2024-08-16T19:22:26.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 576320 4.45 MB/sec execute 525 sec latency 15.906 ms 2024-08-16T19:22:27.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:27 smithi155 ceph-mon[37413]: pgmap v374: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:27.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:27 smithi161 ceph-mon[37576]: pgmap v374: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:27.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 578202 4.46 MB/sec execute 526 sec latency 21.696 ms 2024-08-16T19:22:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:27 smithi062 ceph-mon[32993]: pgmap v374: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:28.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 579663 4.47 MB/sec execute 527 sec latency 13.451 ms 2024-08-16T19:22:29.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:29 smithi155 ceph-mon[37413]: pgmap v375: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:29.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:29 smithi161 ceph-mon[37576]: pgmap v375: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:29.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 581444 4.47 MB/sec execute 528 sec latency 22.504 ms 2024-08-16T19:22:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:29 smithi062 ceph-mon[32993]: pgmap v375: 129 pgs: 129 active+clean; 478 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-08-16T19:22:30.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 583083 4.49 MB/sec execute 529 sec latency 15.165 ms 2024-08-16T19:22:31.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:31 smithi155 ceph-mon[37413]: pgmap v376: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-08-16T19:22:31.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:31 smithi161 ceph-mon[37576]: pgmap v376: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-08-16T19:22:31.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 584899 4.49 MB/sec execute 530 sec latency 22.130 ms 2024-08-16T19:22:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:31 smithi062 ceph-mon[32993]: pgmap v376: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-08-16T19:22:32.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:32 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:32.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 586516 4.51 MB/sec execute 531 sec latency 54.988 ms 2024-08-16T19:22:32.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:32 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:33.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:32 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:33.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:33 smithi161 ceph-mon[37576]: pgmap v377: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-16T19:22:33.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 587932 4.51 MB/sec execute 532 sec latency 21.561 ms 2024-08-16T19:22:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:33 smithi062 ceph-mon[32993]: pgmap v377: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-16T19:22:34.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:33 smithi155 ceph-mon[37413]: pgmap v377: 129 pgs: 129 active+clean; 489 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-16T19:22:34.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:34 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:34.989 INFO:tasks.workunit.client.0.smithi062.stdout: 1 589945 4.52 MB/sec execute 533 sec latency 35.392 ms 2024-08-16T19:22:34.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:34 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:35.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:34 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:35.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:35 smithi161 ceph-mon[37576]: pgmap v378: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-08-16T19:22:35.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 591179 4.53 MB/sec execute 534 sec latency 27.974 ms 2024-08-16T19:22:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:35 smithi062 ceph-mon[32993]: pgmap v378: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-08-16T19:22:36.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:35 smithi155 ceph-mon[37413]: pgmap v378: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-08-16T19:22:36.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 593418 4.54 MB/sec execute 535 sec latency 15.182 ms 2024-08-16T19:22:37.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:37 smithi161 ceph-mon[37576]: pgmap v379: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:37.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 594652 4.54 MB/sec execute 536 sec latency 17.652 ms 2024-08-16T19:22:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:37 smithi062 ceph-mon[32993]: pgmap v379: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:38.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:37 smithi155 ceph-mon[37413]: pgmap v379: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:38.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 596783 4.56 MB/sec execute 537 sec latency 18.700 ms 2024-08-16T19:22:39.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:39 smithi161 ceph-mon[37576]: pgmap v380: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:39.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 597819 4.56 MB/sec execute 538 sec latency 25.193 ms 2024-08-16T19:22:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:39 smithi062 ceph-mon[32993]: pgmap v380: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:39 smithi155 ceph-mon[37413]: pgmap v380: 129 pgs: 129 active+clean; 501 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-16T19:22:40.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 599900 4.57 MB/sec execute 539 sec latency 26.681 ms 2024-08-16T19:22:41.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:41 smithi161 ceph-mon[37576]: pgmap v381: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-08-16T19:22:41.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 601168 4.57 MB/sec execute 540 sec latency 16.883 ms 2024-08-16T19:22:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:41 smithi062 ceph-mon[32993]: pgmap v381: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-08-16T19:22:42.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:41 smithi155 ceph-mon[37413]: pgmap v381: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-08-16T19:22:42.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 603251 4.59 MB/sec execute 541 sec latency 18.490 ms 2024-08-16T19:22:43.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:43 smithi161 ceph-mon[37576]: pgmap v382: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-08-16T19:22:43.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 604630 4.59 MB/sec execute 542 sec latency 16.220 ms 2024-08-16T19:22:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:43 smithi062 ceph-mon[32993]: pgmap v382: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-08-16T19:22:44.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:43 smithi155 ceph-mon[37413]: pgmap v382: 129 pgs: 129 active+clean; 511 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:44 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 606240 4.59 MB/sec execute 543 sec latency 19.523 ms 2024-08-16T19:22:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:22:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:44.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:44 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:22:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.080 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:44 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:45 smithi161 ceph-mon[37576]: pgmap v383: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-08-16T19:22:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:22:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:22:45.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:45.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 607737 4.61 MB/sec execute 544 sec latency 50.074 ms 2024-08-16T19:22:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:45 smithi062 ceph-mon[32993]: pgmap v383: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-08-16T19:22:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:22:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:22:45.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:45 smithi155 ceph-mon[37413]: pgmap v383: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 340 op/s 2024-08-16T19:22:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:22:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:22:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:22:46.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 609155 4.61 MB/sec execute 545 sec latency 21.085 ms 2024-08-16T19:22:47.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:47 smithi155 ceph-mon[37413]: pgmap v384: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:47.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:47 smithi161 ceph-mon[37576]: pgmap v384: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:47.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 611215 4.63 MB/sec execute 546 sec latency 19.552 ms 2024-08-16T19:22:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:47 smithi062 ceph-mon[32993]: pgmap v384: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:48.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 612453 4.63 MB/sec execute 547 sec latency 20.966 ms 2024-08-16T19:22:49.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:49 smithi161 ceph-mon[37576]: pgmap v385: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:49.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 614663 4.64 MB/sec execute 548 sec latency 19.283 ms 2024-08-16T19:22:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:49 smithi062 ceph-mon[32993]: pgmap v385: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:50.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:49 smithi155 ceph-mon[37413]: pgmap v385: 129 pgs: 129 active+clean; 520 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-08-16T19:22:50.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:50 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:50.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 615847 4.65 MB/sec execute 549 sec latency 50.721 ms 2024-08-16T19:22:50.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:50 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:51.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:50 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:51.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:51 smithi161 ceph-mon[37576]: pgmap v386: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-08-16T19:22:51.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 618071 4.66 MB/sec execute 550 sec latency 22.885 ms 2024-08-16T19:22:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:51 smithi062 ceph-mon[32993]: pgmap v386: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-08-16T19:22:52.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:51 smithi155 ceph-mon[37413]: pgmap v386: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-08-16T19:22:52.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 619190 4.66 MB/sec execute 551 sec latency 16.312 ms 2024-08-16T19:22:53.887 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:53 smithi161 ceph-mon[37576]: pgmap v387: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-16T19:22:53.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 621172 4.67 MB/sec execute 552 sec latency 17.439 ms 2024-08-16T19:22:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:53 smithi062 ceph-mon[32993]: pgmap v387: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-16T19:22:54.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:53 smithi155 ceph-mon[37413]: pgmap v387: 129 pgs: 129 active+clean; 530 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-16T19:22:54.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 622451 4.67 MB/sec execute 553 sec latency 13.238 ms 2024-08-16T19:22:54.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:54 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:55.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:54 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:55.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:54 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:55.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 624426 4.68 MB/sec execute 554 sec latency 33.933 ms 2024-08-16T19:22:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:55 smithi062 ceph-mon[32993]: pgmap v388: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-08-16T19:22:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:55 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:56.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:55 smithi155 ceph-mon[37413]: pgmap v388: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-08-16T19:22:56.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:55 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:55 smithi161 ceph-mon[37576]: pgmap v388: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 361 op/s 2024-08-16T19:22:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:55 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:22:56.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 625809 4.69 MB/sec execute 555 sec latency 16.802 ms 2024-08-16T19:22:57.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 627844 4.70 MB/sec execute 556 sec latency 19.348 ms 2024-08-16T19:22:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:57 smithi062 ceph-mon[32993]: pgmap v389: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:22:58.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:57 smithi155 ceph-mon[37413]: pgmap v389: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:22:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:57 smithi161 ceph-mon[37576]: pgmap v389: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:22:58.990 INFO:tasks.workunit.client.0.smithi062.stdout: 1 629349 4.71 MB/sec execute 557 sec latency 13.567 ms 2024-08-16T19:22:59.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 631072 4.71 MB/sec execute 558 sec latency 17.947 ms 2024-08-16T19:22:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:22:59 smithi062 ceph-mon[32993]: pgmap v390: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:23:00.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:22:59 smithi155 ceph-mon[37413]: pgmap v390: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:23:00.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:22:59 smithi161 ceph-mon[37576]: pgmap v390: 129 pgs: 129 active+clean; 540 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-08-16T19:23:00.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 632621 4.72 MB/sec execute 559 sec latency 52.438 ms 2024-08-16T19:23:01.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 633197 4.72 MB/sec execute 560 sec latency 72.965 ms 2024-08-16T19:23:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:01 smithi062 ceph-mon[32993]: pgmap v391: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 364 op/s 2024-08-16T19:23:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:01 smithi155 ceph-mon[37413]: pgmap v391: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 364 op/s 2024-08-16T19:23:02.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:01 smithi161 ceph-mon[37576]: pgmap v391: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 364 op/s 2024-08-16T19:23:02.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 634584 4.72 MB/sec execute 561 sec latency 55.319 ms 2024-08-16T19:23:03.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 636126 4.73 MB/sec execute 562 sec latency 15.704 ms 2024-08-16T19:23:03.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:03 smithi062 ceph-mon[32993]: pgmap v392: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-08-16T19:23:04.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:03 smithi155 ceph-mon[37413]: pgmap v392: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-08-16T19:23:04.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:03 smithi161 ceph-mon[37576]: pgmap v392: 129 pgs: 129 active+clean; 550 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2024-08-16T19:23:04.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 637369 4.73 MB/sec execute 563 sec latency 29.308 ms 2024-08-16T19:23:05.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 639474 4.75 MB/sec execute 564 sec latency 56.285 ms 2024-08-16T19:23:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:05 smithi062 ceph-mon[32993]: pgmap v393: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-08-16T19:23:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:05 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:05.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:05 smithi062 ceph-mon[32993]: osdmap e106: 12 total, 12 up, 12 in 2024-08-16T19:23:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:05 smithi155 ceph-mon[37413]: pgmap v393: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-08-16T19:23:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:05 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:05 smithi155 ceph-mon[37413]: osdmap e106: 12 total, 12 up, 12 in 2024-08-16T19:23:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:05 smithi161 ceph-mon[37576]: pgmap v393: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-08-16T19:23:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:05 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:05 smithi161 ceph-mon[37576]: osdmap e106: 12 total, 12 up, 12 in 2024-08-16T19:23:06.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 640541 4.75 MB/sec execute 565 sec latency 17.716 ms 2024-08-16T19:23:07.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 642808 4.76 MB/sec execute 566 sec latency 32.960 ms 2024-08-16T19:23:07.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:07 smithi062 ceph-mon[32993]: pgmap v395: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:08.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:07 smithi155 ceph-mon[37413]: pgmap v395: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:08.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:07 smithi161 ceph-mon[37576]: pgmap v395: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:08.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 643910 4.76 MB/sec execute 567 sec latency 53.639 ms 2024-08-16T19:23:09.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 645994 4.77 MB/sec execute 568 sec latency 45.521 ms 2024-08-16T19:23:09.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:09 smithi062 ceph-mon[32993]: pgmap v396: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:10.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:09 smithi155 ceph-mon[37413]: pgmap v396: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:10.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:09 smithi161 ceph-mon[37576]: pgmap v396: 129 pgs: 129 active+clean; 558 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 321 op/s 2024-08-16T19:23:10.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 647172 4.78 MB/sec execute 569 sec latency 57.629 ms 2024-08-16T19:23:11.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 648959 4.78 MB/sec execute 570 sec latency 23.834 ms 2024-08-16T19:23:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:11 smithi062 ceph-mon[32993]: pgmap v397: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-08-16T19:23:11.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:11 smithi062 ceph-mon[32993]: osdmap e107: 12 total, 12 up, 12 in 2024-08-16T19:23:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:11 smithi155 ceph-mon[37413]: pgmap v397: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-08-16T19:23:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:11 smithi155 ceph-mon[37413]: osdmap e107: 12 total, 12 up, 12 in 2024-08-16T19:23:12.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:11 smithi161 ceph-mon[37576]: pgmap v397: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-08-16T19:23:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:11 smithi161 ceph-mon[37576]: osdmap e107: 12 total, 12 up, 12 in 2024-08-16T19:23:12.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 650486 4.79 MB/sec execute 571 sec latency 15.316 ms 2024-08-16T19:23:13.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 652340 4.80 MB/sec execute 572 sec latency 22.490 ms 2024-08-16T19:23:13.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:13 smithi062 ceph-mon[32993]: pgmap v399: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-08-16T19:23:14.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:13 smithi155 ceph-mon[37413]: pgmap v399: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-08-16T19:23:14.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:13 smithi161 ceph-mon[37576]: pgmap v399: 129 pgs: 129 active+clean; 568 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-08-16T19:23:14.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 653915 4.81 MB/sec execute 573 sec latency 13.188 ms 2024-08-16T19:23:15.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 655266 4.81 MB/sec execute 574 sec latency 53.378 ms 2024-08-16T19:23:15.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:15 smithi062 ceph-mon[32993]: pgmap v400: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-08-16T19:23:16.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:15 smithi155 ceph-mon[37413]: pgmap v400: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-08-16T19:23:16.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:15 smithi161 ceph-mon[37576]: pgmap v400: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-08-16T19:23:16.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 657226 4.82 MB/sec execute 575 sec latency 27.958 ms 2024-08-16T19:23:17.939 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:17 smithi161 ceph-mon[37576]: pgmap v401: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:17.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 658436 4.83 MB/sec execute 576 sec latency 18.824 ms 2024-08-16T19:23:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:17 smithi062 ceph-mon[32993]: pgmap v401: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:18.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:17 smithi155 ceph-mon[37413]: pgmap v401: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:18.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 660354 4.83 MB/sec execute 577 sec latency 15.137 ms 2024-08-16T19:23:19.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 661519 4.84 MB/sec execute 578 sec latency 14.472 ms 2024-08-16T19:23:19.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:19 smithi062 ceph-mon[32993]: pgmap v402: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:20.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:19 smithi155 ceph-mon[37413]: pgmap v402: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:20.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:19 smithi161 ceph-mon[37576]: pgmap v402: 129 pgs: 129 active+clean; 578 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 286 op/s 2024-08-16T19:23:20.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 663425 4.85 MB/sec execute 579 sec latency 38.379 ms 2024-08-16T19:23:21.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 664671 4.85 MB/sec execute 580 sec latency 17.538 ms 2024-08-16T19:23:21.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:21 smithi062 ceph-mon[32993]: pgmap v403: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-08-16T19:23:22.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:21 smithi155 ceph-mon[37413]: pgmap v403: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-08-16T19:23:22.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:21 smithi161 ceph-mon[37576]: pgmap v403: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-08-16T19:23:22.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 666462 4.85 MB/sec execute 581 sec latency 21.026 ms 2024-08-16T19:23:23.991 INFO:tasks.workunit.client.0.smithi062.stdout: 1 668045 4.87 MB/sec execute 582 sec latency 11.365 ms 2024-08-16T19:23:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:23 smithi062 ceph-mon[32993]: pgmap v404: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-08-16T19:23:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:23 smithi155 ceph-mon[37413]: pgmap v404: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-08-16T19:23:24.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:23 smithi161 ceph-mon[37576]: pgmap v404: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-08-16T19:23:24.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 669314 4.87 MB/sec execute 583 sec latency 23.601 ms 2024-08-16T19:23:25.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 671375 4.88 MB/sec execute 584 sec latency 54.096 ms 2024-08-16T19:23:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:25 smithi062 ceph-mon[32993]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-08-16T19:23:26.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:25 smithi155 ceph-mon[37413]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-08-16T19:23:26.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:25 smithi161 ceph-mon[37576]: pgmap v405: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 357 op/s 2024-08-16T19:23:26.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 672549 4.88 MB/sec execute 585 sec latency 56.351 ms 2024-08-16T19:23:27.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 674761 4.90 MB/sec execute 586 sec latency 25.033 ms 2024-08-16T19:23:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:27 smithi062 ceph-mon[32993]: pgmap v406: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:28.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:27 smithi155 ceph-mon[37413]: pgmap v406: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:28.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:27 smithi161 ceph-mon[37576]: pgmap v406: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:28.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 675859 4.89 MB/sec execute 587 sec latency 17.303 ms 2024-08-16T19:23:29.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 677196 4.89 MB/sec execute 588 sec latency 63.707 ms 2024-08-16T19:23:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:29 smithi062 ceph-mon[32993]: pgmap v407: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:30.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:29 smithi155 ceph-mon[37413]: pgmap v407: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:30.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:29 smithi161 ceph-mon[37576]: pgmap v407: 129 pgs: 129 active+clean; 601 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-08-16T19:23:30.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 678747 4.91 MB/sec execute 589 sec latency 14.166 ms 2024-08-16T19:23:31.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 680225 4.91 MB/sec execute 590 sec latency 52.866 ms 2024-08-16T19:23:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:31 smithi062 ceph-mon[32993]: pgmap v408: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 358 op/s 2024-08-16T19:23:32.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:31 smithi155 ceph-mon[37413]: pgmap v408: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 358 op/s 2024-08-16T19:23:32.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:31 smithi161 ceph-mon[37576]: pgmap v408: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 358 op/s 2024-08-16T19:23:32.994 INFO:tasks.workunit.client.0.smithi062.stdout: 1 682230 4.92 MB/sec execute 591 sec latency 12.693 ms 2024-08-16T19:23:33.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 683466 4.92 MB/sec execute 592 sec latency 15.865 ms 2024-08-16T19:23:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:33 smithi062 ceph-mon[32993]: pgmap v409: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-08-16T19:23:34.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:33 smithi155 ceph-mon[37413]: pgmap v409: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-08-16T19:23:34.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:33 smithi161 ceph-mon[37576]: pgmap v409: 129 pgs: 129 active+clean; 611 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-08-16T19:23:34.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 685508 4.93 MB/sec execute 593 sec latency 17.397 ms 2024-08-16T19:23:35.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 686621 4.93 MB/sec execute 594 sec latency 14.047 ms 2024-08-16T19:23:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:35 smithi062 ceph-mon[32993]: pgmap v410: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:35 smithi155 ceph-mon[37413]: pgmap v410: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:35 smithi161 ceph-mon[37576]: pgmap v410: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:36.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 688412 4.94 MB/sec execute 595 sec latency 154.320 ms 2024-08-16T19:23:37.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 689756 4.95 MB/sec execute 596 sec latency 18.575 ms 2024-08-16T19:23:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:37 smithi062 ceph-mon[32993]: pgmap v411: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:37 smithi155 ceph-mon[37413]: pgmap v411: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:37 smithi161 ceph-mon[37576]: pgmap v411: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:38.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 691198 4.95 MB/sec execute 597 sec latency 15.874 ms 2024-08-16T19:23:39.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 692848 4.96 MB/sec execute 598 sec latency 41.274 ms 2024-08-16T19:23:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:39 smithi062 ceph-mon[32993]: pgmap v412: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:40.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:39 smithi155 ceph-mon[37413]: pgmap v412: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:40.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:39 smithi161 ceph-mon[37576]: pgmap v412: 129 pgs: 129 active+clean; 622 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.1 MiB/s wr, 230 op/s 2024-08-16T19:23:40.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 694000 4.96 MB/sec execute 599 sec latency 18.708 ms 2024-08-16T19:23:41.992 INFO:tasks.workunit.client.0.smithi062.stdout: 1 cleanup 600 sec 2024-08-16T19:23:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:41 smithi062 ceph-mon[32993]: pgmap v413: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:41 smithi155 ceph-mon[37413]: pgmap v413: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:41 smithi161 ceph-mon[37576]: pgmap v413: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.6 MiB/s wr, 347 op/s 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: 0 cleanup 600 sec 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: Operation Count AvgLat MaxLat 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: ---------------------------------------- 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: NTCreateX 95000 3.508 28073.184 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: Close 69810 0.002 0.082 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: Rename 4026 11.669 19387.813 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: Unlink 19176 1.709 60.315 2024-08-16T19:23:42.271 INFO:tasks.workunit.client.0.smithi062.stdout: Deltree 2 99.665 143.727 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Mkdir 1 0.005 0.005 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Qpathinfo 86157 1.167 8179.552 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Qfileinfo 15094 0.001 0.025 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Qfsinfo 15779 0.162 5.325 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Sfileinfo 7754 0.018 8.866 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Find 33290 0.614 14096.988 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: WriteX 47316 0.699 27692.679 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: ReadX 149042 0.027 154.310 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: LockX 310 0.499 3.656 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: UnlockX 310 0.464 3.356 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: Flush 6666 3.647 98.859 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-08-16T19:23:42.272 INFO:tasks.workunit.client.0.smithi062.stdout:Throughput 4.95992 MB/sec 1 clients 1 procs max_latency=28073.189 ms 2024-08-16T19:23:42.273 INFO:tasks.workunit.client.0.smithi062.stderr:+ dbench 10 2024-08-16T19:23:42.273 INFO:tasks.workunit.client.0.smithi062.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-16T19:23:42.273 INFO:tasks.workunit.client.0.smithi062.stdout: 2024-08-16T19:23:42.273 INFO:tasks.workunit.client.0.smithi062.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-16T19:23:42.273 INFO:tasks.workunit.client.0.smithi062.stdout:8 of 10 processes prepared for launch 0 sec 2024-08-16T19:23:42.372 INFO:tasks.workunit.client.0.smithi062.stdout:10 of 10 processes prepared for launch 0 sec 2024-08-16T19:23:42.373 INFO:tasks.workunit.client.0.smithi062.stdout:releasing clients 2024-08-16T19:23:43.341 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-08-16T19:23:43.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:23:43.372 INFO:tasks.workunit.client.0.smithi062.stdout: 10 380 135.65 MB/sec warmup 1 sec latency 62.378 ms 2024-08-16T19:23:43.617 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:23:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:43 smithi062 ceph-mon[32993]: pgmap v414: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-08-16T19:23:44.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:43 smithi155 ceph-mon[37413]: pgmap v414: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-08-16T19:23:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:43 smithi161 ceph-mon[37576]: pgmap v414: 129 pgs: 129 active+clean; 632 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-08-16T19:23:44.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 -- 172.21.15.62:0/1353413986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301068d0 msgr2=0x7fa930106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 --2- 172.21.15.62:0/1353413986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301068d0 0x7fa930106cb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009900 tx=0x7fa92c02a410 comp rx=0 tx=0).stop 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 -- 172.21.15.62:0/1353413986 shutdown_connections 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 --2- 172.21.15.62:0/1353413986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9301007a0 0x7fa93010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 --2- 172.21.15.62:0/1353413986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9300ffe00 0x7fa930100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 --2- 172.21.15.62:0/1353413986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301068d0 0x7fa930106cb0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 -- 172.21.15.62:0/1353413986 >> 172.21.15.62:0/1353413986 conn(0x7fa930076f40 msgr2=0x7fa9300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 -- 172.21.15.62:0/1353413986 shutdown_connections 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.223+0000 7fa937408640 1 -- 172.21.15.62:0/1353413986 wait complete. 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 Processor -- start 2024-08-16T19:23:44.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 -- start start 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9300ffe00 0x7fa9301981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111dc0 con 0x7fa9301007a0 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111c40 con 0x7fa9300ffe00 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.224+0000 7fa937408640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111f40 con 0x7fa9301068d0 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9359a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9359a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:38656/0 (socket says 172.21.15.62:38656) 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9359a5640 1 -- 172.21.15.62:0/3624905637 learned_addr learned my addr 172.21.15.62:0/3624905637 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60292/0 (socket says 172.21.15.62:60292) 2024-08-16T19:23:44.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 -- 172.21.15.62:0/3624905637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 msgr2=0x7fa93019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 -- 172.21.15.62:0/3624905637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9300ffe00 msgr2=0x7fa9301981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9300ffe00 0x7fa9301981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 -- 172.21.15.62:0/3624905637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93019d180 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9359a5640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9349a3640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e9f0 tx=0x7fa92400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92400ccc0 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa924005020 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240049e0 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.225+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93019d470 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.226+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa930100f10 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.227+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa930100260 con 0x7fa9301007a0 2024-08-16T19:23:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.228+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa924005560 con 0x7fa9301007a0 2024-08-16T19:23:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.229+0000 7fa9227fc640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 0x7fa91c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.229+0000 7fa9351a4640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 0x7fa91c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.229+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fa9240cf8d0 con 0x7fa9301007a0 2024-08-16T19:23:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.230+0000 7fa9351a4640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 0x7fa91c0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0096c0 tx=0x7fa92c000bc0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.233+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa9240051d0 con 0x7fa9301007a0 2024-08-16T19:23:44.372 INFO:tasks.workunit.client.0.smithi062.stdout: 10 805 103.63 MB/sec warmup 2 sec latency 595.578 ms 2024-08-16T19:23:44.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.495+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa93002d5e0 con 0x7fa9301007a0 2024-08-16T19:23:44.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.497+0000 7fa9227fc640 1 -- 172.21.15.62:0/3624905637 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11990 (secure 0 0 0) 0x7fa92409c0a0 con 0x7fa9301007a0 2024-08-16T19:23:44.498 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:23:44.498 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"btime":"2024-08-16T19:23:43:700551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:23:43.193520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:23:44.498 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.499+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 msgr2=0x7fa91c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.499+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 0x7fa91c0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0096c0 tx=0x7fa92c000bc0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.499+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 msgr2=0x7fa930198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.499+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e9f0 tx=0x7fa92400eeb0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 shutdown_connections 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa91c0a2cf0 0x7fa91c0a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9301068d0 0x7fa93019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9301007a0 0x7fa930198710 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 --2- 172.21.15.62:0/3624905637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9300ffe00 0x7fa9301981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 >> 172.21.15.62:0/3624905637 conn(0x7fa930076f40 msgr2=0x7fa9301050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 shutdown_connections 2024-08-16T19:23:44.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:44.500+0000 7fa937408640 1 -- 172.21.15.62:0/3624905637 wait complete. 2024-08-16T19:23:44.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:44 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:44.833 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:44 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3624905637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:45.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:44 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:45.082 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:44 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3624905637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:45.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:44 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:45.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:44 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3624905637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:45.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 1320 82.41 MB/sec warmup 3 sec latency 1595.608 ms 2024-08-16T19:23:45.403 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-08-16T19:23:45.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-08-16T19:23:45.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:45 smithi062 ceph-mon[32993]: pgmap v415: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-08-16T19:23:45.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:23:45.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:23:45.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:23:45.716 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:23:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:23:46.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:45 smithi155 ceph-mon[37413]: pgmap v415: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-08-16T19:23:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:23:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:23:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:23:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:23:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:45 smithi161 ceph-mon[37576]: pgmap v415: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.6 MiB/s wr, 346 op/s 2024-08-16T19:23:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:23:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:23:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:23:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:23:46.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 -- 172.21.15.62:0/2876487563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8168100b50 msgr2=0x7f8168100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2876487563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8168100b50 0x7f8168100f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8158009860 tx=0x7f815802a3d0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 -- 172.21.15.62:0/2876487563 shutdown_connections 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2876487563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f816810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2876487563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81681022a0 0x7f81681026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2876487563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8168100b50 0x7f8168100f30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 -- 172.21.15.62:0/2876487563 >> 172.21.15.62:0/2876487563 conn(0x7f81680fc500 msgr2=0x7f81680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 -- 172.21.15.62:0/2876487563 shutdown_connections 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.367+0000 7f816d8f6640 1 -- 172.21.15.62:0/2876487563 wait complete. 2024-08-16T19:23:46.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 Processor -- start 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 -- start start 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f8168103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110cb0 con 0x7f81681022a0 2024-08-16T19:23:46.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110b30 con 0x7f8168100b50 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f816d8f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8168110e30 con 0x7f8168102c20 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.368+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:46136/0 (socket says 172.21.15.62:46136) 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60304/0 (socket says 172.21.15.62:60304) 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f8166ffd640 1 -- 172.21.15.62:0/2737113331 learned_addr learned my addr 172.21.15.62:0/2737113331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f81677fe640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f8168103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 -- 172.21.15.62:0/2737113331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 msgr2=0x7f8168103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f8168103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 -- 172.21.15.62:0/2737113331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 msgr2=0x7f8168103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 -- 172.21.15.62:0/2737113331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81681a6660 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f81677fe640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f8168103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f8166ffd640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f815ffff640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f815000e9f0 tx=0x7f815000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f816c8f4640 1 -- 172.21.15.62:0/2737113331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815000ccc0 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81681a6950 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f816c8f4640 1 -- 172.21.15.62:0/2737113331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8150010910 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.369+0000 7f816c8f4640 1 -- 172.21.15.62:0/2737113331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150018960 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.370+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a6e90 con 0x7f81681022a0 2024-08-16T19:23:46.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.372+0000 7f816c8f4640 1 -- 172.21.15.62:0/2737113331 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8150010ac0 con 0x7f81681022a0 2024-08-16T19:23:46.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2138 65.49 MB/sec warmup 4 sec latency 2595.638 ms 2024-08-16T19:23:46.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.372+0000 7f816fcf4740 1 Processor -- start 2024-08-16T19:23:46.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.372+0000 7f816fcf4740 1 -- start start 2024-08-16T19:23:46.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556fcb6ffa70 con 0x556fcb79aa50 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556fcb697780 con 0x556fcb79a130 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f816fcf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556fcb6fba30 con 0x556fcb7a5310 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:46150/0 (socket says 172.21.15.62:46150) 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 -- 172.21.15.62:0/3174836969 learned_addr learned my addr 172.21.15.62:0/3174836969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f8166ffd640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f815ffff640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 -- 172.21.15.62:0/3174836969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 msgr2=0x556fcb7a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 -- 172.21.15.62:0/3174836969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 msgr2=0x556fcb7a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f81677fe640 1 -- 172.21.15.62:0/3174836969 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fcb471b90 con 0x556fcb79a130 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.373+0000 7f8166ffd640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f81677fe640 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f816000e870 tx=0x7f81600095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ad84d0e522d278 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f816c8f4640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 0x7f81540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f815dffb640 1 -- 172.21.15.62:0/3174836969 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816000ddd0 con 0x556fcb79a130 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f8166ffd640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 0x7f81540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f815dffb640 1 -- 172.21.15.62:0/3174836969 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816001c070 con 0x556fcb79a130 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f815dffb640 1 -- 172.21.15.62:0/3174836969 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160016700 con 0x556fcb79a130 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.374+0000 7f816c8f4640 1 -- 172.21.15.62:0/2737113331 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f81500cfbd0 con 0x7f81681022a0 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 -- 172.21.15.62:0/3174836969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 msgr2=0x556fcb79a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f8166ffd640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 0x7f81540a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x556fcb797520 tx=0x7f81580058b0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f816000e870 tx=0x7f81600095c0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 -- 172.21.15.62:0/3174836969 shutdown_connections 2024-08-16T19:23:46.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 --2- 172.21.15.62:0/3174836969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb79a510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 -- 172.21.15.62:0/3174836969 >> 172.21.15.62:0/3174836969 conn(0x556fcb7966c0 msgr2=0x556fcb796aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:46.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.375+0000 7f816fcf4740 1 -- 172.21.15.62:0/3174836969 shutdown_connections 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.376+0000 7f816fcf4740 1 -- 172.21.15.62:0/3174836969 wait complete. 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.376+0000 7f816fcf4740 1 -- start start 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.376+0000 7f816fcf4740 1 Processor -- start 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79aa50 0x556fcb7a2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb7a5310 0x556fcb7a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x556fcb7a7ee0 con 0x556fcb7a5310 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x556fcb6ba990 con 0x556fcb79aa50 2024-08-16T19:23:46.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556fcb7a8060 con 0x556fcb79a130 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f816fcf4740 10 client.0 _ll_register_callbacks cb 0x556fcb40f090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:38680/0 (socket says 172.21.15.62:38680) 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 -- 172.21.15.62:0/1167947400 learned_addr learned my addr 172.21.15.62:0/1167947400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f81677fe640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb7a5310 0x556fcb7a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f815ffff640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79aa50 0x556fcb7a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 -- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79aa50 msgr2=0x556fcb7a2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79aa50 0x556fcb7a2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 -- 172.21.15.62:0/1167947400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb7a5310 msgr2=0x556fcb7a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb7a5310 0x556fcb7a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.377+0000 7f8166ffd640 1 -- 172.21.15.62:0/1167947400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fcb6b96c0 con 0x556fcb79a130 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f8166ffd640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8158004540 tx=0x7f815802a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71a78af9968fa2b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f8146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580303a0 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f8158030540 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816001c070 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8158030740 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f816fcf4740 10 client.25645 fetch_fsmap learned FSMap version 65 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.378+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556fcb7b3b70 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816001c070 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 65) ==== 8382+0+0 (secure 0 0 0) 0x7f815802f710 con 0x556fcb79a130 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 10 client.25645 fetch_fsmap finished waiting for FSMap version 65 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 10 client.25645 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 20 client.25645 populate_metadata read hostname 'smithi062' 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb7b5030 0x556fcb7b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 4 client.25645 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:23:46.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fcb70d1a0 con 0x556fcb7b5030 2024-08-16T19:23:46.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.379+0000 7f815ffff640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb7b5030 0x556fcb7b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.380+0000 7f8146ffd640 10 client.25645 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:46.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.380+0000 7f815ffff640 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb7b5030 0x556fcb7b74d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b145999bdc54d5 server_cookie=d07512572a3d3876 in_seq=0 out_seq=0 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f8146ffd640 1 -- 172.21.15.62:0/1167947400 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fcb70d1a0 con 0x556fcb7b5030 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f8146ffd640 10 client.25645 handle_command_reply: tid=0 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 client.25645 shutdown 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 msgr2=0x556fcb7a21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8158004540 tx=0x7f815802a3d0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 shutdown_connections 2024-08-16T19:23:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb7b5030 0x556fcb7b74d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb7a5310 0x556fcb7a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79aa50 0x556fcb7a2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1167947400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7a21e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f8146ffd640 0 client.25645 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 >> 172.21.15.62:0/1167947400 conn(0x556fcb7966c0 msgr2=0x556fcb79c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 shutdown_connections 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 1 -- 172.21.15.62:0/1167947400 wait complete. 2024-08-16T19:23:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.383+0000 7f816fcf4740 20 client.25645 trim_cache size 0 max 16384 2024-08-16T19:23:46.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 Processor -- start 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 -- start start 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb7a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81681b1060 con 0x556fcb79aa50 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81681b2c00 con 0x556fcb79a130 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f816fcf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f81681b1730 con 0x556fcb7a5310 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60346/0 (socket says 172.21.15.62:60346) 2024-08-16T19:23:46.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f815ffff640 1 -- 172.21.15.62:0/1447868130 learned_addr learned my addr 172.21.15.62:0/1447868130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f8166ffd640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb7a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.401+0000 7f81677fe640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 -- 172.21.15.62:0/1447868130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 msgr2=0x556fcb7bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 -- 172.21.15.62:0/1447868130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 msgr2=0x556fcb7a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb7a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 -- 172.21.15.62:0/1447868130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fcb797770 con 0x556fcb79aa50 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f8166ffd640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb7a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f81677fe640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815ffff640 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8150098780 tx=0x7f81500d3710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3bda65f62c898 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815d7fa640 1 -- 172.21.15.62:0/1447868130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815009b710 con 0x556fcb79aa50 2024-08-16T19:23:46.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815d7fa640 1 -- 172.21.15.62:0/1447868130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815009bdb0 con 0x556fcb79aa50 2024-08-16T19:23:46.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.402+0000 7f815d7fa640 1 -- 172.21.15.62:0/1447868130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815009b200 con 0x556fcb79aa50 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- 172.21.15.62:0/1447868130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 msgr2=0x556fcb7bcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8150098780 tx=0x7f81500d3710 comp rx=0 tx=0).stop 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- 172.21.15.62:0/1447868130 shutdown_connections 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb7a5310 0x556fcb7bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7bcdf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 --2- 172.21.15.62:0/1447868130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb79a130 0x556fcb7a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- 172.21.15.62:0/1447868130 >> 172.21.15.62:0/1447868130 conn(0x556fcb7966c0 msgr2=0x556fcb7a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- 172.21.15.62:0/1447868130 shutdown_connections 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- 172.21.15.62:0/1447868130 wait complete. 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.403+0000 7f816fcf4740 1 -- start start 2024-08-16T19:23:46.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 Processor -- start 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb7a5310 0x556fcb7a9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f81681b2530 con 0x556fcb79aa50 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f81681b1e30 con 0x556fcb7a5310 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f81681af060 con 0x556fcb79a130 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f816fcf4740 10 client.0 _ll_register_callbacks cb 0x7f8160009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f8166ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:38688/0 (socket says 172.21.15.62:38688) 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f815ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60350/0 (socket says 172.21.15.62:60350) 2024-08-16T19:23:46.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f8166ffd640 1 -- 172.21.15.62:0/710684637 learned_addr learned my addr 172.21.15.62:0/710684637 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.404+0000 7f81677fe640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb7a5310 0x556fcb7a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 -- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb7a5310 msgr2=0x556fcb7a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb7a5310 0x556fcb7a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 -- 172.21.15.62:0/710684637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 msgr2=0x556fcb7a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 -- 172.21.15.62:0/710684637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fcb81edc0 con 0x556fcb79a130 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8166ffd640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f815802fb80 tx=0x7f8158005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf92e03b894ce82 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8146ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158064060 con 0x556fcb79a130 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f815802f770 con 0x556fcb79a130 2024-08-16T19:23:46.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f815009bdb0 con 0x556fcb79a130 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f815806b050 con 0x556fcb79a130 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f816fcf4740 10 client.25651 fetch_fsmap learned FSMap version 65 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.405+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556fcb81ff00 con 0x556fcb79a130 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815009bdb0 con 0x556fcb79a130 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 65) ==== 8382+0+0 (secure 0 0 0) 0x7f81580047c0 con 0x556fcb79a130 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 10 client.25651 fetch_fsmap finished waiting for FSMap version 65 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 10 client.25651 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 20 client.25651 populate_metadata read hostname 'smithi062' 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb820690 0x556fcb822ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:46.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 4 client.25651 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-08-16T19:23:46.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x556fcb7074f0 con 0x556fcb820690 2024-08-16T19:23:46.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.406+0000 7f815ffff640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb820690 0x556fcb822ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:46.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.407+0000 7f815ffff640 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb820690 0x556fcb822ab0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d221ea4a19dbe8 server_cookie=ae3e4020b8d1ead0 in_seq=0 out_seq=0 2024-08-16T19:23:46.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:46.407+0000 7f8146ffd640 10 client.25651 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:46.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:46 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:47.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:46 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:46 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:47.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2175 52.90 MB/sec warmup 5 sec latency 3595.672 ms 2024-08-16T19:23:47.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f8146ffd640 1 -- 172.21.15.62:0/710684637 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556fcb7074f0 con 0x556fcb820690 2024-08-16T19:23:47.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f8146ffd640 10 client.25651 handle_command_reply: tid=0 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f816fcf4740 1 client.25651 shutdown 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 msgr2=0x556fcb7adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f815802fb80 tx=0x7f8158005fa0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.406+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 shutdown_connections 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x556fcb820690 0x556fcb822ab0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fcb7a5310 0x556fcb7a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556fcb79aa50 0x556fcb7a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 --2- 172.21.15.62:0/710684637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556fcb79a130 0x556fcb7adb60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f8146ffd640 0 client.25651 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 >> 172.21.15.62:0/710684637 conn(0x556fcb7966c0 msgr2=0x556fcb79c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 shutdown_connections 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 1 -- 172.21.15.62:0/710684637 wait complete. 2024-08-16T19:23:47.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816fcf4740 20 client.25651 trim_cache size 0 max 16384 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 66, 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 46, 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "2d683cbd": { 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 46, 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:23:47.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 130.0, 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:47.410 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 msgr2=0x7f81540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 0x7f81540a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x556fcb797520 tx=0x7f81580058b0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 msgr2=0x7f81681039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:47.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f815000e9f0 tx=0x7f815000eeb0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 shutdown_connections 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f81540a2cf0 0x7f81540a51b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8168102c20 0x7f8168103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81681022a0 0x7f81681039c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 --2- 172.21.15.62:0/2737113331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8168100b50 0x7f8168103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.407+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 >> 172.21.15.62:0/2737113331 conn(0x7f81680fc500 msgr2=0x7f8168069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.408+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 shutdown_connections 2024-08-16T19:23:47.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:47.408+0000 7f816d8f6640 1 -- 172.21.15.62:0/2737113331 wait complete. 2024-08-16T19:23:47.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:47 smithi155 ceph-mon[37413]: pgmap v416: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:47.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:47 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:47 smithi062 ceph-mon[32993]: pgmap v416: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:47 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:48.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d683cbd', quiesce took 4.7 seconds. Will release after: 5.3 2024-08-16T19:23:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:47 smithi161 ceph-mon[37576]: pgmap v416: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:47 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:23:48.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 44.08 MB/sec warmup 6 sec latency 4595.711 ms 2024-08-16T19:23:49.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 37.78 MB/sec warmup 7 sec latency 5595.788 ms 2024-08-16T19:23:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:49 smithi062 ceph-mon[32993]: pgmap v417: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:50.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:49 smithi155 ceph-mon[37413]: pgmap v417: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:49 smithi161 ceph-mon[37576]: pgmap v417: 129 pgs: 129 active+clean; 642 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-08-16T19:23:50.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 33.06 MB/sec warmup 8 sec latency 6595.836 ms 2024-08-16T19:23:51.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 29.39 MB/sec warmup 9 sec latency 7595.936 ms 2024-08-16T19:23:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:51 smithi062 ceph-mon[32993]: pgmap v418: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-08-16T19:23:52.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:51 smithi155 ceph-mon[37413]: pgmap v418: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-08-16T19:23:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:51 smithi161 ceph-mon[37576]: pgmap v418: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2024-08-16T19:23:52.373 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 26.45 MB/sec warmup 10 sec latency 8596.043 ms 2024-08-16T19:23:53.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:23:53.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 24.04 MB/sec warmup 11 sec latency 9596.709 ms 2024-08-16T19:23:53.641 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:23:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:53 smithi062 ceph-mon[32993]: pgmap v419: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-08-16T19:23:54.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:53 smithi155 ceph-mon[37413]: pgmap v419: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-08-16T19:23:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:53 smithi161 ceph-mon[37576]: pgmap v419: 129 pgs: 129 active+clean; 655 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 234 op/s 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.125+0000 7f987c9e2640 1 -- 172.21.15.62:0/1038992466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9878100b50 msgr2=0x7f9878100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.125+0000 7f987c9e2640 1 --2- 172.21.15.62:0/1038992466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9878100b50 0x7f9878100f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9868009900 tx=0x7f986802a480 comp rx=0 tx=0).stop 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 -- 172.21.15.62:0/1038992466 shutdown_connections 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 --2- 172.21.15.62:0/1038992466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 0x7f987810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 --2- 172.21.15.62:0/1038992466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98781022a0 0x7f98781026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 --2- 172.21.15.62:0/1038992466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9878100b50 0x7f9878100f30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 -- 172.21.15.62:0/1038992466 >> 172.21.15.62:0/1038992466 conn(0x7f98780fc500 msgr2=0x7f98780fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 -- 172.21.15.62:0/1038992466 shutdown_connections 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 -- 172.21.15.62:0/1038992466 wait complete. 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.126+0000 7f987c9e2640 1 Processor -- start 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 -- start start 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9878100b50 0x7f9878198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 0x7f987819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9878110cb0 con 0x7f98781022a0 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9878110b30 con 0x7f9878100b50 2024-08-16T19:23:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987c9e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9878110e30 con 0x7f9878102c20 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34378/0 (socket says 172.21.15.62:34378) 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 -- 172.21.15.62:0/445115202 learned_addr learned my addr 172.21.15.62:0/445115202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f987659c640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9878100b50 0x7f9878198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9876d9d640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 0x7f987819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 -- 172.21.15.62:0/445115202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 msgr2=0x7f987819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 0x7f987819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 -- 172.21.15.62:0/445115202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9878100b50 msgr2=0x7f9878198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9878100b50 0x7f9878198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 -- 172.21.15.62:0/445115202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987819d1d0 con 0x7f98781022a0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.127+0000 7f9875d9b640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c920 tx=0x7f986c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.128+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c007c70 con 0x7f98781022a0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.128+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986c007e10 con 0x7f98781022a0 2024-08-16T19:23:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.128+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987819d4c0 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.128+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c018890 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.128+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98781a4da0 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.129+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9878100f30 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.130+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f986c018a50 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.131+0000 7f98677fe640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 0x7f98600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.131+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f986c0cf970 con 0x7f98781022a0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.131+0000 7f987659c640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 0x7f98600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.131+0000 7f987659c640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 0x7f98600a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f98680058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:54.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.135+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f986c014030 con 0x7f98781022a0 2024-08-16T19:23:54.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.361+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9878109e00 con 0x7f98781022a0 2024-08-16T19:23:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.362+0000 7f98677fe640 1 -- 172.21.15.62:0/445115202 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11993 (secure 0 0 0) 0x7f986c09c1a0 con 0x7f98781022a0 2024-08-16T19:23:54.363 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:23:54.363 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":66,"btime":"2024-08-16T19:23:46:723518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:23:45.885085+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:23:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 66 2024-08-16T19:23:54.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 msgr2=0x7f98600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:54.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 0x7f98600a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f98680058d0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 msgr2=0x7f9878198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c920 tx=0x7f986c00cde0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 shutdown_connections 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f98600a2cf0 0x7f98600a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.364+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9878102c20 0x7f987819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.365+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98781022a0 0x7f9878198760 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.365+0000 7f987c9e2640 1 --2- 172.21.15.62:0/445115202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9878100b50 0x7f9878198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.365+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 >> 172.21.15.62:0/445115202 conn(0x7f98780fc500 msgr2=0x7f9878069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.365+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 shutdown_connections 2024-08-16T19:23:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:54.365+0000 7f987c9e2640 1 -- 172.21.15.62:0/445115202 wait complete. 2024-08-16T19:23:54.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 22.04 MB/sec warmup 12 sec latency 10596.748 ms 2024-08-16T19:23:54.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:54 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/445115202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:55.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:54 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/445115202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:55.129 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 2d683cbd --release --await' 2024-08-16T19:23:55.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 2d683cbd --release --await 2024-08-16T19:23:55.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:54 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/445115202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:23:55.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2176 20.34 MB/sec warmup 13 sec latency 11596.799 ms 2024-08-16T19:23:55.454 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:23:55.896 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:55 smithi062 ceph-mon[32993]: pgmap v420: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-08-16T19:23:56.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 -- 172.21.15.62:0/1230038090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa841022a0 msgr2=0x7ffa841026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/1230038090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa841022a0 0x7ffa841026e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009900 tx=0x7ffa6c02a480 comp rx=0 tx=0).stop 2024-08-16T19:23:56.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 -- 172.21.15.62:0/1230038090 shutdown_connections 2024-08-16T19:23:56.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/1230038090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa84102c20 0x7ffa8410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/1230038090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa841022a0 0x7ffa841026e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/1230038090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84100b50 0x7ffa84100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 -- 172.21.15.62:0/1230038090 >> 172.21.15.62:0/1230038090 conn(0x7ffa840fc500 msgr2=0x7ffa840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 -- 172.21.15.62:0/1230038090 shutdown_connections 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.987+0000 7ffa8a39a640 1 -- 172.21.15.62:0/1230038090 wait complete. 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 Processor -- start 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 -- start start 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa841022a0 0x7ffa841995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84102c20 0x7ffa841a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84110c70 con 0x7ffa84100b50 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84110af0 con 0x7ffa84102c20 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa8a39a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84110df0 con 0x7ffa841022a0 2024-08-16T19:23:56.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34396/0 (socket says 172.21.15.62:34396) 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 -- 172.21.15.62:0/3807448449 learned_addr learned my addr 172.21.15.62:0/3807448449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa88937640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84102c20 0x7ffa841a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa841022a0 0x7ffa841995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 -- 172.21.15.62:0/3807448449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa841022a0 msgr2=0x7ffa841995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa841022a0 0x7ffa841995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 -- 172.21.15.62:0/3807448449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84102c20 msgr2=0x7ffa841a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84102c20 0x7ffa841a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.988+0000 7ffa83fff640 1 -- 172.21.15.62:0/3807448449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa841a4640 con 0x7ffa84100b50 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa83fff640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c920 tx=0x7ffa7400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa81ffb640 1 -- 172.21.15.62:0/3807448449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007c70 con 0x7ffa84100b50 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa841a4810 con 0x7ffa84100b50 2024-08-16T19:23:56.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa81ffb640 1 -- 172.21.15.62:0/3807448449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74007e10 con 0x7ffa84100b50 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa81ffb640 1 -- 172.21.15.62:0/3807448449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74018890 con 0x7ffa84100b50 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.989+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa841a4d20 con 0x7ffa84100b50 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.991+0000 7ffa81ffb640 1 -- 172.21.15.62:0/3807448449 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa74018a70 con 0x7ffa84100b50 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.991+0000 7ffa8c798740 1 Processor -- start 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 -- start start 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78bb2b0 0x561fe78c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561fe7820c60 con 0x561fe78ba990 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561fe77b8b90 con 0x561fe78bb2b0 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa8c798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fe781cc20 con 0x561fe78c5b70 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34404/0 (socket says 172.21.15.62:34404) 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 -- 172.21.15.62:0/196695957 learned_addr learned my addr 172.21.15.62:0/196695957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa83fff640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78bb2b0 0x561fe78c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa81ffb640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 0x7ffa700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 -- 172.21.15.62:0/196695957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 msgr2=0x561fe78c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 -- 172.21.15.62:0/196695957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78bb2b0 msgr2=0x561fe78c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78bb2b0 0x561fe78c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa88937640 1 -- 172.21.15.62:0/196695957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe7530580 con 0x561fe78ba990 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.992+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa81ffb640 1 -- 172.21.15.62:0/3807448449 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ffa740cfa40 con 0x7ffa84100b50 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa88937640 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009e60 tx=0x7ffa7c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674a801cfb199194 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa88937640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 0x7ffa700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa7a7fc640 1 -- 172.21.15.62:0/196695957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c021070 con 0x561fe78ba990 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa7a7fc640 1 -- 172.21.15.62:0/196695957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c0113e0 con 0x561fe78ba990 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.993+0000 7ffa7a7fc640 1 -- 172.21.15.62:0/196695957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c016070 con 0x561fe78ba990 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- 172.21.15.62:0/196695957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 msgr2=0x561fe78bad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009e60 tx=0x7ffa7c010740 comp rx=0 tx=0).stop 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa88937640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 0x7ffa700a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c01e000 tx=0x7ffa7c021000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- 172.21.15.62:0/196695957 shutdown_connections 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78bb2b0 0x561fe78c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 --2- 172.21.15.62:0/196695957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78bad70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- 172.21.15.62:0/196695957 >> 172.21.15.62:0/196695957 conn(0x561fe78b6f20 msgr2=0x561fe78b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- 172.21.15.62:0/196695957 shutdown_connections 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- 172.21.15.62:0/196695957 wait complete. 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.994+0000 7ffa8c798740 1 -- start start 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 Processor -- start 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 0x561fe78c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561fe78ccb50 con 0x561fe78bb2b0 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561fe77be7c0 con 0x561fe78ba990 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561fe78cccd0 con 0x561fe78c5b70 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa8c798740 10 client.0 _ll_register_callbacks cb 0x561fe77c7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34406/0 (socket says 172.21.15.62:34406) 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2449368781 learned_addr learned my addr 172.21.15.62:0/2449368781 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.995+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 0x561fe78c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa88937640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2449368781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 msgr2=0x561fe78c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 msgr2=0x561fe78c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 0x561fe78c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2449368781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe75fa960 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa88937640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 0x561fe78c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5e3a6f7725a815 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa637fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c035030 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ffa6c0048c0 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c0113e0 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c04d050 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa8c798740 10 client.16092 fetch_fsmap learned FSMap version 66 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561fe78d8b90 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.996+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c0113e0 con 0x561fe78bb2b0 2024-08-16T19:23:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 66) ==== 8386+0+0 (secure 0 0 0) 0x7ffa6c031540 con 0x561fe78bb2b0 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 10 client.16092 fetch_fsmap finished waiting for FSMap version 66 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 10 client.16092 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 20 client.16092 populate_metadata read hostname 'smithi062' 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe78da050 0x561fe78dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 4 client.16092 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe782e1d0 con 0x561fe78da050 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.997+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe78da050 0x561fe78dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe78da050 0x561fe78dc480 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed8a226a599ea1a server_cookie=939d1d53116e55f3 in_seq=0 out_seq=0 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa637fe640 10 client.16092 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa637fe640 1 -- 172.21.15.62:0/2449368781 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe782e1d0 con 0x561fe78da050 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa637fe640 10 client.16092 handle_command_reply: tid=0 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa8c798740 1 client.16092 shutdown 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 msgr2=0x561fe78c2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.998+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c02a9a0 tx=0x7ffa6c0052d0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 shutdown_connections 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe78da050 0x561fe78dc480 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78bb2b0 0x561fe78c2fa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2449368781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78ba990 0x561fe78c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 >> 172.21.15.62:0/2449368781 conn(0x561fe78b6f20 msgr2=0x561fe78bcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa637fe640 0 client.16092 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:56.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 shutdown_connections 2024-08-16T19:23:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 1 -- 172.21.15.62:0/2449368781 wait complete. 2024-08-16T19:23:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:55.999+0000 7ffa8c798740 20 client.16092 trim_cache size 0 max 16384 2024-08-16T19:23:56.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.026+0000 7ffa8c798740 1 Processor -- start 2024-08-16T19:23:56.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.026+0000 7ffa8c798740 1 -- start start 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.026+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.026+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78cc520 0x561fe78e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa8c798740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841acca0 con 0x561fe78ba990 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa8c798740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841ae840 con 0x561fe78c5b70 2024-08-16T19:23:56.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa8c798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841ad370 con 0x561fe78cc520 2024-08-16T19:23:56.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54120/0 (socket says 172.21.15.62:54120) 2024-08-16T19:23:56.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa7bfff640 1 -- 172.21.15.62:0/714482254 learned_addr learned my addr 172.21.15.62:0/714482254 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:56.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.027+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 -- 172.21.15.62:0/714482254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78cc520 msgr2=0x561fe78e2290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78cc520 0x561fe78e2290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 -- 172.21.15.62:0/714482254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 msgr2=0x561fe78c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78c1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 -- 172.21.15.62:0/714482254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe78c82a0 con 0x561fe78c5b70 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c036040 tx=0x7ffa6c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45590c101ceafa5b server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa79ffb640 1 -- 172.21.15.62:0/714482254 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c056070 con 0x561fe78c5b70 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa79ffb640 1 -- 172.21.15.62:0/714482254 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c03f070 con 0x561fe78c5b70 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.028+0000 7ffa79ffb640 1 -- 172.21.15.62:0/714482254 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0035f0 con 0x561fe78c5b70 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 -- 172.21.15.62:0/714482254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 msgr2=0x561fe78e1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c036040 tx=0x7ffa6c02a480 comp rx=0 tx=0).stop 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 -- 172.21.15.62:0/714482254 shutdown_connections 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78cc520 0x561fe78e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78c5b70 0x561fe78e1d50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 --2- 172.21.15.62:0/714482254 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78c1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 -- 172.21.15.62:0/714482254 >> 172.21.15.62:0/714482254 conn(0x561fe78b6f20 msgr2=0x561fe78c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.029+0000 7ffa8c798740 1 -- 172.21.15.62:0/714482254 shutdown_connections 2024-08-16T19:23:56.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.030+0000 7ffa8c798740 1 -- 172.21.15.62:0/714482254 wait complete. 2024-08-16T19:23:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.030+0000 7ffa8c798740 1 -- start start 2024-08-16T19:23:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 Processor -- start 2024-08-16T19:23:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 0x561fe78d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841ae170 con 0x561fe78ba990 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841ada70 con 0x561fe78d1bd0 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa841aac70 con 0x561fe78c5b70 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.031+0000 7ffa8c798740 10 client.0 _ll_register_callbacks cb 0x7ffa6c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.032+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.032+0000 7ffa7bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:42966/0 (socket says 172.21.15.62:42966) 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.032+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2486418969 learned_addr learned my addr 172.21.15.62:0/2486418969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.032+0000 7ffa88937640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 0x561fe78d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 msgr2=0x561fe78d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 0x561fe78d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2486418969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 msgr2=0x561fe78d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa7bfff640 1 -- 172.21.15.62:0/2486418969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe78d2830 con 0x561fe78c5b70 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.033+0000 7ffa88937640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 0x561fe78d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.034+0000 7ffa7bfff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009900 tx=0x7ffa6c004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f486ae839bfc34 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.034+0000 7ffa637fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.034+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c035070 con 0x561fe78c5b70 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.034+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c03f070 con 0x561fe78c5b70 2024-08-16T19:23:56.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.034+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ffa6c031b00 con 0x561fe78c5b70 2024-08-16T19:23:56.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.035+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c056040 con 0x561fe78c5b70 2024-08-16T19:23:56.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.035+0000 7ffa8c798740 10 client.25666 fetch_fsmap learned FSMap version 66 2024-08-16T19:23:56.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.035+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561fe7941b30 con 0x561fe78c5b70 2024-08-16T19:23:56.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.035+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa6c03f070 con 0x561fe78c5b70 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 66) ==== 8386+0+0 (secure 0 0 0) 0x7ffa6c059030 con 0x561fe78c5b70 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 10 client.25666 fetch_fsmap finished waiting for FSMap version 66 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 10 client.25666 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 20 client.25666 populate_metadata read hostname 'smithi062' 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe7942260 0x561fe7944680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 4 client.25666 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d683cbd", "release": true, "await": true}] 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.036+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d683cbd", "release": true, "await": true}) -- 0x561fe78b21d0 con 0x561fe7942260 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.037+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe7942260 0x561fe7944680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.037+0000 7ffa637fe640 10 client.25666 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:56.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.037+0000 7ffa83fff640 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe7942260 0x561fe7944680 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f202d56b6826acb server_cookie=484324473efa223e in_seq=0 out_seq=0 2024-08-16T19:23:56.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:55 smithi155 ceph-mon[37413]: pgmap v420: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa637fe640 1 -- 172.21.15.62:0/2486418969 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561fe78b21d0 con 0x561fe7942260 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa637fe640 10 client.25666 handle_command_reply: tid=0 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 client.25666 shutdown 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 msgr2=0x561fe78d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009900 tx=0x7ffa6c004f40 comp rx=0 tx=0).stop 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 shutdown_connections 2024-08-16T19:23:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561fe7942260 0x561fe7944680 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fe78d1bd0 0x561fe78d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561fe78c5b70 0x561fe78d1640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa637fe640 0 client.25666 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 --2- 172.21.15.62:0/2486418969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561fe78ba990 0x561fe78d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 >> 172.21.15.62:0/2486418969 conn(0x561fe78b6f20 msgr2=0x561fe78b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 shutdown_connections 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 1 -- 172.21.15.62:0/2486418969 wait complete. 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.131+0000 7ffa8c798740 20 client.25666 trim_cache size 0 max 16384 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:23:56.133 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 66, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 48, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "2d683cbd": { 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 48, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 130.0, 2024-08-16T19:23:56.134 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:23:56.135 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 msgr2=0x7ffa700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 0x7ffa700a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c01e000 tx=0x7ffa7c021000 comp rx=0 tx=0).stop 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 msgr2=0x7ffa84199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c920 tx=0x7ffa7400cde0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 shutdown_connections 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffa700a2cf0 0x7ffa700a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa84102c20 0x7ffa841a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa841022a0 0x7ffa841995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 --2- 172.21.15.62:0/3807448449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa84100b50 0x7ffa84199080 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:23:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 >> 172.21.15.62:0/3807448449 conn(0x7ffa840fc500 msgr2=0x7ffa84106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:23:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 shutdown_connections 2024-08-16T19:23:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:23:56.132+0000 7ffa8a39a640 1 -- 172.21.15.62:0/3807448449 wait complete. 2024-08-16T19:23:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:55 smithi161 ceph-mon[37576]: pgmap v420: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-08-16T19:23:56.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2243 19.11 MB/sec warmup 14 sec latency 12408.623 ms 2024-08-16T19:23:57.214 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d683cbd', total seconds elapsed: 13.9 2024-08-16T19:23:57.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 2768 21.35 MB/sec warmup 15 sec latency 101.360 ms 2024-08-16T19:23:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:57 smithi062 ceph-mon[32993]: pgmap v421: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:23:58.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:57 smithi155 ceph-mon[37413]: pgmap v421: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:23:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:57 smithi161 ceph-mon[37576]: pgmap v421: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:23:58.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 3410 23.20 MB/sec warmup 16 sec latency 50.452 ms 2024-08-16T19:23:59.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 3649 22.64 MB/sec warmup 17 sec latency 97.134 ms 2024-08-16T19:24:00.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:23:59 smithi155 ceph-mon[37413]: pgmap v422: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:24:00.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:23:59 smithi161 ceph-mon[37576]: pgmap v422: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:24:00.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:23:59 smithi062 ceph-mon[32993]: pgmap v422: 129 pgs: 129 active+clean; 651 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 2.1 MiB/s wr, 149 op/s 2024-08-16T19:24:00.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 3859 21.50 MB/sec warmup 18 sec latency 112.326 ms 2024-08-16T19:24:01.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 4116 20.59 MB/sec warmup 19 sec latency 152.372 ms 2024-08-16T19:24:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:01 smithi155 ceph-mon[37413]: pgmap v423: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2024-08-16T19:24:02.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:01 smithi161 ceph-mon[37576]: pgmap v423: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2024-08-16T19:24:02.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:01 smithi062 ceph-mon[32993]: pgmap v423: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 223 op/s 2024-08-16T19:24:02.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 4349 19.95 MB/sec warmup 20 sec latency 173.091 ms 2024-08-16T19:24:03.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 4455 19.54 MB/sec warmup 21 sec latency 156.864 ms 2024-08-16T19:24:04.084 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:03 smithi155 ceph-mon[37413]: pgmap v424: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:24:04.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:03 smithi161 ceph-mon[37576]: pgmap v424: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:24:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:03 smithi062 ceph-mon[32993]: pgmap v424: 129 pgs: 129 active+clean; 663 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.3 MiB/s wr, 110 op/s 2024-08-16T19:24:04.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 4987 20.32 MB/sec warmup 22 sec latency 128.102 ms 2024-08-16T19:24:05.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 5697 20.32 MB/sec warmup 23 sec latency 630.785 ms 2024-08-16T19:24:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:05 smithi155 ceph-mon[37413]: pgmap v425: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-08-16T19:24:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:05 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:06.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:05 smithi155 ceph-mon[37413]: osdmap e108: 12 total, 12 up, 12 in 2024-08-16T19:24:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:05 smithi161 ceph-mon[37576]: pgmap v425: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-08-16T19:24:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:05 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:06.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:05 smithi161 ceph-mon[37576]: osdmap e108: 12 total, 12 up, 12 in 2024-08-16T19:24:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:05 smithi062 ceph-mon[32993]: pgmap v425: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-08-16T19:24:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:05 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:05 smithi062 ceph-mon[32993]: osdmap e108: 12 total, 12 up, 12 in 2024-08-16T19:24:06.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 6308 21.62 MB/sec warmup 24 sec latency 1630.812 ms 2024-08-16T19:24:07.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 6690 21.45 MB/sec warmup 25 sec latency 2630.846 ms 2024-08-16T19:24:08.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:07 smithi155 ceph-mon[37413]: pgmap v427: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:08.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:07 smithi161 ceph-mon[37576]: pgmap v427: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:07 smithi062 ceph-mon[32993]: pgmap v427: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:08.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 7125 21.87 MB/sec warmup 26 sec latency 3630.880 ms 2024-08-16T19:24:09.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 7398 21.58 MB/sec warmup 27 sec latency 4630.912 ms 2024-08-16T19:24:10.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:09 smithi155 ceph-mon[37413]: pgmap v428: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:10.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:09 smithi161 ceph-mon[37576]: pgmap v428: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:10.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:09 smithi062 ceph-mon[32993]: pgmap v428: 129 pgs: 129 active+clean; 684 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.3 MiB/s wr, 461 op/s 2024-08-16T19:24:10.374 INFO:tasks.workunit.client.0.smithi062.stdout: 10 7721 21.12 MB/sec warmup 28 sec latency 5630.945 ms 2024-08-16T19:24:11.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:10 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:11.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:10 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:10 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:11.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 8336 21.28 MB/sec warmup 29 sec latency 6630.976 ms 2024-08-16T19:24:12.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:11 smithi155 ceph-mon[37413]: pgmap v429: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 613 op/s 2024-08-16T19:24:12.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:11 smithi155 ceph-mon[37413]: osdmap e109: 12 total, 12 up, 12 in 2024-08-16T19:24:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:11 smithi161 ceph-mon[37576]: pgmap v429: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 613 op/s 2024-08-16T19:24:12.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:11 smithi161 ceph-mon[37576]: osdmap e109: 12 total, 12 up, 12 in 2024-08-16T19:24:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:11 smithi062 ceph-mon[32993]: pgmap v429: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 23 MiB/s wr, 613 op/s 2024-08-16T19:24:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:11 smithi062 ceph-mon[32993]: osdmap e109: 12 total, 12 up, 12 in 2024-08-16T19:24:12.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 8987 21.77 MB/sec warmup 30 sec latency 7631.021 ms 2024-08-16T19:24:13.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 9760 22.60 MB/sec warmup 31 sec latency 8631.063 ms 2024-08-16T19:24:14.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:13 smithi155 ceph-mon[37413]: pgmap v431: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 25 MiB/s wr, 302 op/s 2024-08-16T19:24:14.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:13 smithi161 ceph-mon[37576]: pgmap v431: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 25 MiB/s wr, 302 op/s 2024-08-16T19:24:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:13 smithi062 ceph-mon[32993]: pgmap v431: 129 pgs: 129 active+clean; 869 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 25 MiB/s wr, 302 op/s 2024-08-16T19:24:14.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 10308 23.24 MB/sec warmup 32 sec latency 9631.097 ms 2024-08-16T19:24:15.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 10646 22.82 MB/sec warmup 33 sec latency 10030.356 ms 2024-08-16T19:24:15.786 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-16T19:24:15.787 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-16T19:24:15.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:24:16.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:15 smithi155 ceph-mon[37413]: pgmap v432: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-08-16T19:24:16.102 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:16.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:15 smithi161 ceph-mon[37576]: pgmap v432: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-08-16T19:24:16.229 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:15 smithi062 ceph-mon[32993]: pgmap v432: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 23 MiB/s wr, 441 op/s 2024-08-16T19:24:16.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 11171 22.89 MB/sec warmup 34 sec latency 6616.337 ms 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 -- 172.21.15.62:0/2130249407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 msgr2=0x7ffb6c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 --2- 172.21.15.62:0/2130249407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c101d20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009930 tx=0x7ffb6002a420 comp rx=0 tx=0).stop 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 -- 172.21.15.62:0/2130249407 shutdown_connections 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 --2- 172.21.15.62:0/2130249407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c102260 0x7ffb6c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 --2- 172.21.15.62:0/2130249407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c101d20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 --2- 172.21.15.62:0/2130249407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c1078c0 0x7ffb6c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 -- 172.21.15.62:0/2130249407 >> 172.21.15.62:0/2130249407 conn(0x7ffb6c0fd620 msgr2=0x7ffb6c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 -- 172.21.15.62:0/2130249407 shutdown_connections 2024-08-16T19:24:16.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.614+0000 7ffb721f7640 1 -- 172.21.15.62:0/2130249407 wait complete. 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 Processor -- start 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 -- start start 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c102260 0x7ffb6c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c1078c0 0x7ffb6c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c111cc0 con 0x7ffb6c1018c0 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c111b40 con 0x7ffb6c1078c0 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb721f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c111e40 con 0x7ffb6c102260 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb6b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42216/0 (socket says 172.21.15.62:42216) 2024-08-16T19:24:16.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb6b7fe640 1 -- 172.21.15.62:0/473208614 learned_addr learned my addr 172.21.15.62:0/473208614 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:24:16.927 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"btime":"2024-08-16T19:24:09:764063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:24:09.567303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:24:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb6bfff640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c1078c0 0x7ffb6c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.615+0000 7ffb6affd640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c102260 0x7ffb6c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 -- 172.21.15.62:0/473208614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c102260 msgr2=0x7ffb6c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:16.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c102260 0x7ffb6c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 -- 172.21.15.62:0/473208614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c1078c0 msgr2=0x7ffb6c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:16.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c1078c0 0x7ffb6c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 -- 172.21.15.62:0/473208614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6c1a6680 con 0x7ffb6c1018c0 2024-08-16T19:24:16.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb6b7fe640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:16.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00cd80 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c1a6970 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c010910 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0188f0 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.616+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c1a6ed0 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.617+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c101d20 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.619+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb5c010ac0 con 0x7ffb6c1018c0 2024-08-16T19:24:16.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.620+0000 7ffb68ff9640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 0x7ffb540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.620+0000 7ffb6affd640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 0x7ffb540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.620+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7ffb5c0cfbd0 con 0x7ffb6c1018c0 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.620+0000 7ffb6affd640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 0x7ffb540a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000c00 tx=0x7ffb60005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.625+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffb5c09c400 con 0x7ffb6c1018c0 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.881+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb6c075750 con 0x7ffb6c1018c0 2024-08-16T19:24:16.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.882+0000 7ffb68ff9640 1 -- 172.21.15.62:0/473208614 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+12000 (secure 0 0 0) 0x7ffb5c09bd00 con 0x7ffb6c1018c0 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 68 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 msgr2=0x7ffb540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 0x7ffb540a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000c00 tx=0x7ffb60005a20 comp rx=0 tx=0).stop 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 msgr2=0x7ffb6c074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9f0 tx=0x7ffb5c00eeb0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 shutdown_connections 2024-08-16T19:24:16.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb540a2cf0 0x7ffb540a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6c1078c0 0x7ffb6c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6c102260 0x7ffb6c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 --2- 172.21.15.62:0/473208614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb6c1018c0 0x7ffb6c074220 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.884+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 >> 172.21.15.62:0/473208614 conn(0x7ffb6c0fd620 msgr2=0x7ffb6c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.885+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 shutdown_connections 2024-08-16T19:24:16.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:16.885+0000 7ffb721f7640 1 -- 172.21.15.62:0/473208614 wait complete. 2024-08-16T19:24:17.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 11659 22.96 MB/sec warmup 35 sec latency 7616.378 ms 2024-08-16T19:24:17.814 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:17 smithi062 ceph-mon[32993]: pgmap v433: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:17.815 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:17 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/473208614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:17.816 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-16T19:24:17.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-16T19:24:18.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:17 smithi155 ceph-mon[37413]: pgmap v433: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:17 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/473208614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:18.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:17 smithi161 ceph-mon[37576]: pgmap v433: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:18.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:17 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/473208614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:18.147 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:18.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12376 23.09 MB/sec warmup 36 sec latency 8616.816 ms 2024-08-16T19:24:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 -- 172.21.15.62:0/541630836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c301022a0 msgr2=0x7f3c301026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 --2- 172.21.15.62:0/541630836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c301022a0 0x7f3c301026e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009900 tx=0x7f3c2002a480 comp rx=0 tx=0).stop 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 -- 172.21.15.62:0/541630836 shutdown_connections 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 --2- 172.21.15.62:0/541630836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c30102c20 0x7f3c3010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 --2- 172.21.15.62:0/541630836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c301022a0 0x7f3c301026e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 --2- 172.21.15.62:0/541630836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30100b50 0x7f3c30100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 -- 172.21.15.62:0/541630836 >> 172.21.15.62:0/541630836 conn(0x7f3c300fc500 msgr2=0x7f3c300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 -- 172.21.15.62:0/541630836 shutdown_connections 2024-08-16T19:24:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.005+0000 7f3c380a5640 1 -- 172.21.15.62:0/541630836 wait complete. 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 Processor -- start 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 -- start start 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c301022a0 0x7f3c3019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c35e41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 0x7f3c3019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c35e41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42224/0 (socket says 172.21.15.62:42224) 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c35e41640 1 -- 172.21.15.62:0/1793867311 learned_addr learned my addr 172.21.15.62:0/1793867311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30110cb0 con 0x7f3c30100b50 2024-08-16T19:24:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30110b30 con 0x7f3c301022a0 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.006+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c30110e30 con 0x7f3c30102c20 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35640640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c301022a0 0x7f3c3019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c36642640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 0x7f3c3019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 -- 172.21.15.62:0/1793867311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 msgr2=0x7f3c3019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 0x7f3c3019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 -- 172.21.15.62:0/1793867311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c301022a0 msgr2=0x7f3c3019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c301022a0 0x7f3c3019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3019f3b0 con 0x7f3c30100b50 2024-08-16T19:24:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c36642640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 0x7f3c3019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:24:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.007+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9f0 tx=0x7f3c2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.008+0000 7f3c26ffd640 1 -- 172.21.15.62:0/1793867311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00cd80 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.008+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3019f6a0 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.008+0000 7f3c26ffd640 1 -- 172.21.15.62:0/1793867311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2c005500 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.008+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c301a6f80 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.008+0000 7f3c26ffd640 1 -- 172.21.15.62:0/1793867311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c010690 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.011+0000 7f3c26ffd640 1 -- 172.21.15.62:0/1793867311 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c2c0046c0 con 0x7f3c30100b50 2024-08-16T19:24:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.011+0000 7f3c3a4a3740 1 Processor -- start 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 -- start start 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab911dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab914700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561cab86ca70 con 0x561cab907130 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561cab804780 con 0x561cab907a50 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cab868a30 con 0x561cab912310 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c36642640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c36642640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42226/0 (socket says 172.21.15.62:42226) 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c36642640 1 -- 172.21.15.62:0/2873108643 learned_addr learned my addr 172.21.15.62:0/2873108643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.012+0000 7f3c35640640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab914700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c35e41640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab911dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c26ffd640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 0x7f3c1c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c26ffd640 1 -- 172.21.15.62:0/1793867311 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f3c2c0cfc60 con 0x7f3c30100b50 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 0x7f3c1c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 -- 172.21.15.62:0/2873108643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 msgr2=0x561cab914700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab914700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 -- 172.21.15.62:0/2873108643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 msgr2=0x561cab911dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab911dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 -- 172.21.15.62:0/2873108643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cab5deb90 con 0x561cab907130 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c35640640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab914700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c35e41640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab911dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.013+0000 7f3c36642640 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013740 tx=0x7f3c28013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5448cb051cef9b7f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c1bfff640 1 -- 172.21.15.62:0/2873108643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28022070 con 0x561cab907130 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c36642640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 0x7f3c1c0a52b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801f000 tx=0x7f3c2801a000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c1bfff640 1 -- 172.21.15.62:0/2873108643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2801c430 con 0x561cab907130 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c1bfff640 1 -- 172.21.15.62:0/2873108643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28010630 con 0x561cab907130 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2873108643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 msgr2=0x561cab907510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013740 tx=0x7f3c28013c00 comp rx=0 tx=0).stop 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2873108643 shutdown_connections 2024-08-16T19:24:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab914700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab911dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2873108643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab907510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2873108643 >> 172.21.15.62:0/2873108643 conn(0x561cab9036c0 msgr2=0x561cab903aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.014+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2873108643 shutdown_connections 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2873108643 wait complete. 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 -- start start 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 Processor -- start 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907130 0x561cab90f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 0x561cab90f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561cab914ee0 con 0x561cab907a50 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x561cab827990 con 0x561cab907130 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cab915060 con 0x561cab912310 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.015+0000 7f3c3a4a3740 10 client.0 _ll_register_callbacks cb 0x561cab57c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c35640640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 0x561cab90f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c35e41640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907130 0x561cab90f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:43242/0 (socket says 172.21.15.62:43242) 2024-08-16T19:24:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 -- 172.21.15.62:0/2924266644 learned_addr learned my addr 172.21.15.62:0/2924266644 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c35640640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 0x561cab90f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42240/0 (socket says 172.21.15.62:42240) 2024-08-16T19:24:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 -- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907130 msgr2=0x561cab90f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907130 0x561cab90f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 -- 172.21.15.62:0/2924266644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 msgr2=0x561cab90f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 0x561cab90f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 -- 172.21.15.62:0/2924266644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cab8266c0 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c36642640 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013d60 tx=0x7f3c2801ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd76925ef15467c server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28022040 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2801c430 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.016+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f3c28009d40 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3c28036050 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 10 client.25684 fetch_fsmap learned FSMap version 68 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x561cab920ba0 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2801c430 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 68) ==== 8402+0+0 (secure 0 0 0) 0x7f3c28010910 con 0x561cab912310 2024-08-16T19:24:19.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 10 client.25684 fetch_fsmap finished waiting for FSMap version 68 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 10 client.25684 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 20 client.25684 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab922420 0x561cab924880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 4 client.25684 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cab87a1a0 con 0x561cab922420 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.017+0000 7f3c35e41640 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab922420 0x561cab924880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.018+0000 7f3c197fa640 10 client.25684 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.018+0000 7f3c35e41640 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab922420 0x561cab924880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826b48449900429f server_cookie=e413daa396a299d9 in_seq=0 out_seq=0 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c197fa640 1 -- 172.21.15.62:0/2924266644 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cab87a1a0 con 0x561cab922420 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c197fa640 10 client.25684 handle_command_reply: tid=0 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 client.25684 shutdown 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 msgr2=0x561cab90fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013d60 tx=0x7f3c2801ef60 comp rx=0 tx=0).stop 2024-08-16T19:24:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 shutdown_connections 2024-08-16T19:24:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab922420 0x561cab924880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90fc90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907a50 0x561cab90f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c197fa640 0 client.25684 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/2924266644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907130 0x561cab90f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 >> 172.21.15.62:0/2924266644 conn(0x561cab9036c0 msgr2=0x561cab9093f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 shutdown_connections 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/2924266644 wait complete. 2024-08-16T19:24:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.020+0000 7f3c3a4a3740 20 client.25684 trim_cache size 0 max 16384 2024-08-16T19:24:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 Processor -- start 2024-08-16T19:24:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 -- start start 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab9110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab909be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b1020 con 0x561cab907130 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b2c80 con 0x561cab907a50 2024-08-16T19:24:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b16f0 con 0x561cab912310 2024-08-16T19:24:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c35e41640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab9110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c36642640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c36642640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:43252/0 (socket says 172.21.15.62:43252) 2024-08-16T19:24:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c36642640 1 -- 172.21.15.62:0/836936008 learned_addr learned my addr 172.21.15.62:0/836936008 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.042+0000 7f3c35640640 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab909be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 -- 172.21.15.62:0/836936008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 msgr2=0x561cab909be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab909be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 -- 172.21.15.62:0/836936008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 msgr2=0x561cab9110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab9110e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 -- 172.21.15.62:0/836936008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cab901c70 con 0x561cab912310 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c35e41640 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab9110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c36642640 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013570 tx=0x7f3c28013c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d02db58193dc99 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c1b7fe640 1 -- 172.21.15.62:0/836936008 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28033070 con 0x561cab912310 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c1b7fe640 1 -- 172.21.15.62:0/836936008 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28010910 con 0x561cab912310 2024-08-16T19:24:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.043+0000 7f3c1b7fe640 1 -- 172.21.15.62:0/836936008 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28003bd0 con 0x561cab912310 2024-08-16T19:24:19.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/836936008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 msgr2=0x561cab90dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c28013570 tx=0x7f3c28013c00 comp rx=0 tx=0).stop 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/836936008 shutdown_connections 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab912310 0x561cab90dae0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab907a50 0x561cab909be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/836936008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab9110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/836936008 >> 172.21.15.62:0/836936008 conn(0x561cab9036c0 msgr2=0x561cab90f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/836936008 shutdown_connections 2024-08-16T19:24:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.044+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/836936008 wait complete. 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 -- start start 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 Processor -- start 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab91aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab912310 0x561cab916290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b24f0 con 0x561cab907130 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301b1df0 con 0x561cab912310 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3c301aeff0 con 0x561cab907a50 2024-08-16T19:24:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.045+0000 7f3c3a4a3740 10 client.0 _ll_register_callbacks cb 0x7f3c28020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c36642640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab912310 0x561cab916290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:43258/0 (socket says 172.21.15.62:43258) 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 -- 172.21.15.62:0/1486547571 learned_addr learned my addr 172.21.15.62:0/1486547571 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab91aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 -- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab912310 msgr2=0x561cab916290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab912310 0x561cab916290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 -- 172.21.15.62:0/1486547571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 msgr2=0x561cab91aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab91aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 -- 172.21.15.62:0/1486547571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cab916ae0 con 0x561cab907a50 2024-08-16T19:24:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c35640640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c200314e0 tx=0x7f3c20004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a860ca4db382dcce server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.046+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2004a060 con 0x561cab907a50 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f3c20003ac0 con 0x561cab907a50 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c28010910 con 0x561cab907a50 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3c20051050 con 0x561cab907a50 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 10 client.25690 fetch_fsmap learned FSMap version 68 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x561cab995970 con 0x561cab907a50 2024-08-16T19:24:19.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28010910 con 0x561cab907a50 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 68) ==== 8402+0+0 (secure 0 0 0) 0x7f3c2003c440 con 0x561cab907a50 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 10 client.25690 fetch_fsmap finished waiting for FSMap version 68 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 10 client.25690 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 20 client.25690 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab996250 0x561cab998670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 4 client.25690 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.047+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x561cab8744f0 con 0x561cab996250 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.048+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab996250 0x561cab998670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.048+0000 7f3c35e41640 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab996250 0x561cab998670 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b948ff3669b57480 server_cookie=c6e3119aafa882a8 in_seq=0 out_seq=0 2024-08-16T19:24:19.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.048+0000 7f3c197fa640 10 client.25690 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:19.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12897 23.45 MB/sec warmup 37 sec latency 9616.854 ms 2024-08-16T19:24:19.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c197fa640 1 -- 172.21.15.62:0/1486547571 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561cab8744f0 con 0x561cab996250 2024-08-16T19:24:19.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c197fa640 10 client.25690 handle_command_reply: tid=0 2024-08-16T19:24:19.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 client.25690 shutdown 2024-08-16T19:24:19.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 msgr2=0x561cab915d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c200314e0 tx=0x7f3c20004460 comp rx=0 tx=0).stop 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 shutdown_connections 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x561cab996250 0x561cab998670 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cab912310 0x561cab916290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cab907a50 0x561cab915d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 --2- 172.21.15.62:0/1486547571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561cab907130 0x561cab91aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c197fa640 0 client.25690 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 >> 172.21.15.62:0/1486547571 conn(0x561cab9036c0 msgr2=0x561cab901ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 shutdown_connections 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 1 -- 172.21.15.62:0/1486547571 wait complete. 2024-08-16T19:24:19.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.882+0000 7f3c3a4a3740 20 client.25690 trim_cache size 0 max 16384 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 68, 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 50, 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "ea9abd1c": { 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 50, 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:19.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 msgr2=0x7f3c1c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 0x7f3c1c0a52b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801f000 tx=0x7f3c2801a000 comp rx=0 tx=0).stop 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 msgr2=0x7f3c3019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9f0 tx=0x7f3c2c00edf0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 shutdown_connections 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f3c1c0a2df0 0x7f3c1c0a52b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c30102c20 0x7f3c3019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c301022a0 0x7f3c3019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 --2- 172.21.15.62:0/1793867311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c30100b50 0x7f3c3019a400 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 >> 172.21.15.62:0/1793867311 conn(0x7f3c300fc500 msgr2=0x7f3c3006a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 shutdown_connections 2024-08-16T19:24:19.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:19.883+0000 7f3c380a5640 1 -- 172.21.15.62:0/1793867311 wait complete. 2024-08-16T19:24:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:19 smithi155 ceph-mon[37413]: pgmap v434: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:20.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:19 smithi161 ceph-mon[37576]: pgmap v434: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:20.243 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:19 smithi062 ceph-mon[32993]: pgmap v434: 129 pgs: 129 active+clean; 888 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 431 op/s 2024-08-16T19:24:20.375 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 22.83 MB/sec warmup 38 sec latency 9759.116 ms 2024-08-16T19:24:20.561 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ea9abd1c', quiesce took 4.8 seconds. Will release after: 25.2 2024-08-16T19:24:21.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 22.25 MB/sec warmup 39 sec latency 2300.169 ms 2024-08-16T19:24:22.081 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:21 smithi155 ceph-mon[37413]: pgmap v435: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 453 op/s 2024-08-16T19:24:22.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:21 smithi161 ceph-mon[37576]: pgmap v435: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 453 op/s 2024-08-16T19:24:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:21 smithi062 ceph-mon[32993]: pgmap v435: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.3 MiB/s wr, 453 op/s 2024-08-16T19:24:22.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 21.69 MB/sec warmup 40 sec latency 3300.223 ms 2024-08-16T19:24:23.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 21.16 MB/sec warmup 41 sec latency 4300.262 ms 2024-08-16T19:24:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:23 smithi155 ceph-mon[37413]: pgmap v436: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 415 op/s 2024-08-16T19:24:24.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:23 smithi161 ceph-mon[37576]: pgmap v436: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 415 op/s 2024-08-16T19:24:24.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:23 smithi062 ceph-mon[32993]: pgmap v436: 129 pgs: 129 active+clean; 936 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 415 op/s 2024-08-16T19:24:24.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 20.66 MB/sec warmup 42 sec latency 5300.306 ms 2024-08-16T19:24:25.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 20.18 MB/sec warmup 43 sec latency 6300.421 ms 2024-08-16T19:24:26.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:25 smithi155 ceph-mon[37413]: pgmap v437: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 415 op/s 2024-08-16T19:24:26.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:25 smithi161 ceph-mon[37576]: pgmap v437: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 415 op/s 2024-08-16T19:24:26.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:25 smithi062 ceph-mon[32993]: pgmap v437: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.9 MiB/s wr, 415 op/s 2024-08-16T19:24:26.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 19.72 MB/sec warmup 44 sec latency 7300.548 ms 2024-08-16T19:24:27.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 19.28 MB/sec warmup 45 sec latency 8300.606 ms 2024-08-16T19:24:28.049 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:27 smithi062 ceph-mon[32993]: pgmap v438: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:28.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:27 smithi155 ceph-mon[37413]: pgmap v438: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:28.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:27 smithi161 ceph-mon[37576]: pgmap v438: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:28.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 18.86 MB/sec warmup 46 sec latency 9300.710 ms 2024-08-16T19:24:29.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 18.46 MB/sec warmup 47 sec latency 10300.759 ms 2024-08-16T19:24:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:29 smithi155 ceph-mon[37413]: pgmap v439: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:30.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:29 smithi161 ceph-mon[37576]: pgmap v439: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:30.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:29 smithi062 ceph-mon[32993]: pgmap v439: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.8 MiB/s wr, 257 op/s 2024-08-16T19:24:30.376 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 18.07 MB/sec warmup 48 sec latency 11301.072 ms 2024-08-16T19:24:31.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 17.71 MB/sec warmup 49 sec latency 12301.176 ms 2024-08-16T19:24:32.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:31 smithi155 ceph-mon[37413]: pgmap v440: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 284 op/s 2024-08-16T19:24:32.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:31 smithi161 ceph-mon[37576]: pgmap v440: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 284 op/s 2024-08-16T19:24:32.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:31 smithi062 ceph-mon[32993]: pgmap v440: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.1 MiB/s wr, 284 op/s 2024-08-16T19:24:32.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 17.35 MB/sec warmup 50 sec latency 13301.298 ms 2024-08-16T19:24:33.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 17.01 MB/sec warmup 51 sec latency 14301.408 ms 2024-08-16T19:24:34.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:33 smithi155 ceph-mon[37413]: pgmap v441: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-08-16T19:24:34.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:33 smithi161 ceph-mon[37576]: pgmap v441: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-08-16T19:24:34.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:33 smithi062 ceph-mon[32993]: pgmap v441: 129 pgs: 129 active+clean; 947 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-08-16T19:24:34.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 16.68 MB/sec warmup 52 sec latency 15301.461 ms 2024-08-16T19:24:35.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 16.37 MB/sec warmup 53 sec latency 16301.500 ms 2024-08-16T19:24:36.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:35 smithi155 ceph-mon[37413]: pgmap v442: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 3.0 MiB/s wr, 83 op/s 2024-08-16T19:24:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:35 smithi161 ceph-mon[37576]: pgmap v442: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 3.0 MiB/s wr, 83 op/s 2024-08-16T19:24:36.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:35 smithi062 ceph-mon[32993]: pgmap v442: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 3.0 MiB/s wr, 83 op/s 2024-08-16T19:24:36.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 16.07 MB/sec warmup 54 sec latency 17301.581 ms 2024-08-16T19:24:37.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 15.77 MB/sec warmup 55 sec latency 18301.667 ms 2024-08-16T19:24:38.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:37 smithi155 ceph-mon[37413]: pgmap v443: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:37 smithi161 ceph-mon[37576]: pgmap v443: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:38.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:37 smithi062 ceph-mon[32993]: pgmap v443: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:38.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 15.49 MB/sec warmup 56 sec latency 19301.733 ms 2024-08-16T19:24:39.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 15.22 MB/sec warmup 57 sec latency 20301.811 ms 2024-08-16T19:24:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:39 smithi155 ceph-mon[37413]: pgmap v444: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:40.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:39 smithi161 ceph-mon[37576]: pgmap v444: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:40.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:39 smithi062 ceph-mon[32993]: pgmap v444: 129 pgs: 129 active+clean; 953 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 1.9 MiB/s wr, 46 op/s 2024-08-16T19:24:40.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 14.96 MB/sec warmup 58 sec latency 21301.878 ms 2024-08-16T19:24:41.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 14.70 MB/sec warmup 59 sec latency 22301.937 ms 2024-08-16T19:24:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:41 smithi155 ceph-mon[37413]: pgmap v445: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.8 MiB/s wr, 121 op/s 2024-08-16T19:24:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:41 smithi161 ceph-mon[37576]: pgmap v445: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.8 MiB/s wr, 121 op/s 2024-08-16T19:24:42.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:41 smithi062 ceph-mon[32993]: pgmap v445: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 3.8 MiB/s wr, 121 op/s 2024-08-16T19:24:42.384 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 14.46 MB/sec warmup 60 sec latency 23302.035 ms 2024-08-16T19:24:43.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 14.22 MB/sec warmup 61 sec latency 24302.085 ms 2024-08-16T19:24:44.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:43 smithi155 ceph-mon[37413]: pgmap v446: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-08-16T19:24:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:43 smithi161 ceph-mon[37576]: pgmap v446: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-08-16T19:24:44.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:43 smithi062 ceph-mon[32993]: pgmap v446: 129 pgs: 129 active+clean; 962 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.4 MiB/s wr, 94 op/s 2024-08-16T19:24:44.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 13.99 MB/sec warmup 62 sec latency 25302.123 ms 2024-08-16T19:24:45.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 13.77 MB/sec warmup 63 sec latency 26302.222 ms 2024-08-16T19:24:45.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:24:46.063 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:46.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:45 smithi155 ceph-mon[37413]: pgmap v447: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.9 MiB/s wr, 168 op/s 2024-08-16T19:24:46.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:45 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:24:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:45 smithi161 ceph-mon[37576]: pgmap v447: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.9 MiB/s wr, 168 op/s 2024-08-16T19:24:46.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:45 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:24:46.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:45 smithi062 ceph-mon[32993]: pgmap v447: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.9 MiB/s wr, 168 op/s 2024-08-16T19:24:46.190 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:45 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:24:46.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 13.56 MB/sec warmup 64 sec latency 27302.255 ms 2024-08-16T19:24:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.648+0000 7f1490cd4640 1 -- 172.21.15.62:0/1658590982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c100b50 msgr2=0x7f148c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.648+0000 7f1490cd4640 1 --2- 172.21.15.62:0/1658590982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c100b50 0x7f148c100f30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f147c009900 tx=0x7f147c02a480 comp rx=0 tx=0).stop 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 -- 172.21.15.62:0/1658590982 shutdown_connections 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 --2- 172.21.15.62:0/1658590982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c102c20 0x7f148c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 --2- 172.21.15.62:0/1658590982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c1022a0 0x7f148c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 --2- 172.21.15.62:0/1658590982 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c100b50 0x7f148c100f30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 -- 172.21.15.62:0/1658590982 >> 172.21.15.62:0/1658590982 conn(0x7f148c0fc500 msgr2=0x7f148c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 -- 172.21.15.62:0/1658590982 shutdown_connections 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 -- 172.21.15.62:0/1658590982 wait complete. 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 Processor -- start 2024-08-16T19:24:46.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.649+0000 7f1490cd4640 1 -- start start 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c1022a0 0x7f148c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 0x7f148c1a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148a59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148a59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50124/0 (socket says 172.21.15.62:50124) 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148a59c640 1 -- 172.21.15.62:0/3887492210 learned_addr learned my addr 172.21.15.62:0/3887492210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f148c110cb0 con 0x7f148c102c20 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f148c110b30 con 0x7f148c100b50 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f148c110e30 con 0x7f148c1022a0 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 0x7f148c1a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f1483fff640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c1022a0 0x7f148c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 -- 172.21.15.62:0/3887492210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c1022a0 msgr2=0x7f148c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c1022a0 0x7f148c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 -- 172.21.15.62:0/3887492210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 msgr2=0x7f148c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148ad9d640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c1a4620 con 0x7f148c102c20 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.650+0000 7f148a59c640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f148ad9d640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 0x7f148c1a40e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1484009fd0 tx=0x7f148400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484004d20 con 0x7f148c102c20 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c196e50 con 0x7f148c102c20 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c197390 con 0x7f148c102c20 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1484005020 con 0x7f148c102c20 2024-08-16T19:24:46.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.651+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148400f620 con 0x7f148c102c20 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.652+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c100f30 con 0x7f148c102c20 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.653+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1484005520 con 0x7f148c102c20 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.654+0000 7f14837fe640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 0x7f14780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.654+0000 7f148a59c640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 0x7f14780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.654+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f14840d85b0 con 0x7f148c102c20 2024-08-16T19:24:46.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.655+0000 7f148a59c640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 0x7f14780a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f147c0096c0 tx=0x7f147c005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:46.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.658+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f14840a4ef0 con 0x7f148c102c20 2024-08-16T19:24:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.872+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f148c108dc0 con 0x7f148c102c20 2024-08-16T19:24:46.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.872+0000 7f14837fe640 1 -- 172.21.15.62:0/3887492210 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+12000 (secure 0 0 0) 0x7f14840a47f0 con 0x7f148c102c20 2024-08-16T19:24:46.874 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:24:46.874 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"btime":"2024-08-16T19:24:09:764063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:24:09.567303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:24:46.874 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 68 2024-08-16T19:24:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.874+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 msgr2=0x7f14780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.874+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 0x7f14780a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f147c0096c0 tx=0x7f147c005a20 comp rx=0 tx=0).stop 2024-08-16T19:24:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.874+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 msgr2=0x7f148c1a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.874+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 0x7f148c1a40e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1484009fd0 tx=0x7f148400ef10 comp rx=0 tx=0).stop 2024-08-16T19:24:46.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 shutdown_connections 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f14780a2d70 0x7f14780a5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f148c102c20 0x7f148c1a40e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f148c1022a0 0x7f148c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 --2- 172.21.15.62:0/3887492210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f148c100b50 0x7f148c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 >> 172.21.15.62:0/3887492210 conn(0x7f148c0fc500 msgr2=0x7f148c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 shutdown_connections 2024-08-16T19:24:46.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:46.875+0000 7f1490cd4640 1 -- 172.21.15.62:0/3887492210 wait complete. 2024-08-16T19:24:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:46 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:24:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:46 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:24:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:46 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:24:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:46 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:24:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:46 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:24:47.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:46 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:24:47.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:46 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:24:47.212 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:46 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:24:47.213 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:46 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:24:47.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 13.35 MB/sec warmup 65 sec latency 28302.286 ms 2024-08-16T19:24:47.531 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id ea9abd1c --release --await' 2024-08-16T19:24:47.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id ea9abd1c --release --await 2024-08-16T19:24:47.797 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:47 smithi155 ceph-mon[37413]: pgmap v448: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:47 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3887492210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:48.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:47 smithi062 ceph-mon[32993]: pgmap v448: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:48.079 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:47 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3887492210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:47 smithi161 ceph-mon[37576]: pgmap v448: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:47 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3887492210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.332+0000 7fef93d8a640 1 -- 172.21.15.62:0/813305582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c1022a0 msgr2=0x7fef8c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.332+0000 7fef93d8a640 1 --2- 172.21.15.62:0/813305582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c1022a0 0x7fef8c1026e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0099b0 tx=0x7fef7c02c700 comp rx=0 tx=0).stop 2024-08-16T19:24:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.334+0000 7fef93d8a640 1 -- 172.21.15.62:0/813305582 shutdown_connections 2024-08-16T19:24:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.334+0000 7fef93d8a640 1 --2- 172.21.15.62:0/813305582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c102c20 0x7fef8c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.334+0000 7fef93d8a640 1 --2- 172.21.15.62:0/813305582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c1022a0 0x7fef8c1026e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.334+0000 7fef93d8a640 1 --2- 172.21.15.62:0/813305582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c100b50 0x7fef8c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.334+0000 7fef93d8a640 1 -- 172.21.15.62:0/813305582 >> 172.21.15.62:0/813305582 conn(0x7fef8c0fc500 msgr2=0x7fef8c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.335+0000 7fef93d8a640 1 -- 172.21.15.62:0/813305582 shutdown_connections 2024-08-16T19:24:48.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.335+0000 7fef93d8a640 1 -- 172.21.15.62:0/813305582 wait complete. 2024-08-16T19:24:48.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.336+0000 7fef93d8a640 1 Processor -- start 2024-08-16T19:24:48.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 -- start start 2024-08-16T19:24:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 0x7fef8c196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 0x7fef8c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 0x7fef8c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c10fbf0 con 0x7fef8c100b50 2024-08-16T19:24:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c10fa70 con 0x7fef8c102c20 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.337+0000 7fef93d8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c10fd70 con 0x7fef8c1022a0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef92327640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 0x7fef8c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91b26640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 0x7fef8c196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef92327640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 0x7fef8c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:50146/0 (socket says 172.21.15.62:50146) 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef92327640 1 -- 172.21.15.62:0/1758613826 learned_addr learned my addr 172.21.15.62:0/1758613826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 0x7fef8c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 -- 172.21.15.62:0/1758613826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 msgr2=0x7fef8c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 0x7fef8c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 -- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 msgr2=0x7fef8c196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 0x7fef8c196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 -- 172.21.15.62:0/1758613826 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19afe0 con 0x7fef8c1022a0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.338+0000 7fef91325640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 0x7fef8c196570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009230 tx=0x7fef7c005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef91b26640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 0x7fef8c196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:24:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef82ffd640 1 -- 172.21.15.62:0/1758613826 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c044030 con 0x7fef8c1022a0 2024-08-16T19:24:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef82ffd640 1 -- 172.21.15.62:0/1758613826 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c040c20 con 0x7fef8c1022a0 2024-08-16T19:24:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef82ffd640 1 -- 172.21.15.62:0/1758613826 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c033c70 con 0x7fef8c1022a0 2024-08-16T19:24:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c19b270 con 0x7fef8c1022a0 2024-08-16T19:24:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.339+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c19b6d0 con 0x7fef8c1022a0 2024-08-16T19:24:48.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.343+0000 7fef96188740 1 Processor -- start 2024-08-16T19:24:48.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.343+0000 7fef82ffd640 1 -- 172.21.15.62:0/1758613826 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef7c047030 con 0x7fef8c1022a0 2024-08-16T19:24:48.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.343+0000 7fef96188740 1 -- start start 2024-08-16T19:24:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc92157c60 con 0x55cc921f22b0 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc920efb90 con 0x55cc921fcb70 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.344+0000 7fef96188740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc92153c20 con 0x55cc921f1990 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:46412/0 (socket says 172.21.15.62:46412) 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 -- 172.21.15.62:0/3826872780 learned_addr learned my addr 172.21.15.62:0/3826872780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef91325640 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 -- 172.21.15.62:0/3826872780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 msgr2=0x55cc921fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 -- 172.21.15.62:0/3826872780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 msgr2=0x55cc921fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.345+0000 7fef92327640 1 -- 172.21.15.62:0/3826872780 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc91e67580 con 0x55cc921f1990 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef82ffd640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 0x7fef780a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef92327640 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef84022580 tx=0x7fef84022a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234e5d73ce79c566 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef91b26640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 0x7fef780a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef77fff640 1 -- 172.21.15.62:0/3826872780 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004b20 con 0x55cc921f1990 2024-08-16T19:24:48.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef82ffd640 1 -- 172.21.15.62:0/1758613826 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fef7c033a10 con 0x7fef8c1022a0 2024-08-16T19:24:48.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef77fff640 1 -- 172.21.15.62:0/3826872780 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef84004cc0 con 0x55cc921f1990 2024-08-16T19:24:48.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.346+0000 7fef77fff640 1 -- 172.21.15.62:0/3826872780 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef840054c0 con 0x55cc921f1990 2024-08-16T19:24:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.347+0000 7fef91b26640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 0x7fef780a53f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef8800b2c0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.347+0000 7fef96188740 1 -- 172.21.15.62:0/3826872780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 msgr2=0x55cc921f1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.347+0000 7fef96188740 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef84022580 tx=0x7fef84022a40 comp rx=0 tx=0).stop 2024-08-16T19:24:48.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 -- 172.21.15.62:0/3826872780 shutdown_connections 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 --2- 172.21.15.62:0/3826872780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f1d70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 -- 172.21.15.62:0/3826872780 >> 172.21.15.62:0/3826872780 conn(0x55cc921edf20 msgr2=0x55cc921ee300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 -- 172.21.15.62:0/3826872780 shutdown_connections 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 -- 172.21.15.62:0/3826872780 wait complete. 2024-08-16T19:24:48.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 -- start start 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.348+0000 7fef96188740 1 Processor -- start 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921f22b0 0x55cc921f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921fcb70 0x55cc921fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55cc921ff740 con 0x55cc921fcb70 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55cc920f57c0 con 0x55cc921f22b0 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cc921ff8c0 con 0x55cc921f1990 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef96188740 10 client.0 _ll_register_callbacks cb 0x55cc920fef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef91b26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef91b26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:46422/0 (socket says 172.21.15.62:46422) 2024-08-16T19:24:48.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef91b26640 1 -- 172.21.15.62:0/3692110278 learned_addr learned my addr 172.21.15.62:0/3692110278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef92327640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921fcb70 0x55cc921fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.349+0000 7fef91325640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921f22b0 0x55cc921f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 -- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921f22b0 msgr2=0x55cc921f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921f22b0 0x55cc921f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 -- 172.21.15.62:0/3692110278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921fcb70 msgr2=0x55cc921fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921fcb70 0x55cc921fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 -- 172.21.15.62:0/3692110278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc91f31960 con 0x55cc921f1990 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef91b26640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b570 tx=0x7fef88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59d820c9916c498 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef757fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88022070 con 0x55cc921f1990 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fef8801e860 con 0x55cc921f1990 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef84004cc0 con 0x55cc921f1990 2024-08-16T19:24:48.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fef8802e050 con 0x55cc921f1990 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef96188740 10 client.25708 fetch_fsmap learned FSMap version 68 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.350+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55cc9220b770 con 0x55cc921f1990 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef84004cc0 con 0x55cc921f1990 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 68) ==== 8402+0+0 (secure 0 0 0) 0x7fef88010bd0 con 0x55cc921f1990 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 10 client.25708 fetch_fsmap finished waiting for FSMap version 68 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 10 client.25708 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 20 client.25708 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc9220cbd0 0x55cc9220f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 4 client.25708 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc921651d0 con 0x55cc9220cbd0 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.351+0000 7fef91325640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc9220cbd0 0x55cc9220f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.352+0000 7fef91325640 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc9220cbd0 0x55cc9220f030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68eaf3f20ec10dd1 server_cookie=21d61f38d9df95d6 in_seq=0 out_seq=0 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.352+0000 7fef757fa640 10 client.25708 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.352+0000 7fef757fa640 1 -- 172.21.15.62:0/3692110278 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc921651d0 con 0x55cc9220cbd0 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.352+0000 7fef757fa640 10 client.25708 handle_command_reply: tid=0 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.352+0000 7fef96188740 1 client.25708 shutdown 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 msgr2=0x55cc921f99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b570 tx=0x7fef88004730 comp rx=0 tx=0).stop 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 shutdown_connections 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc9220cbd0 0x55cc9220f030 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921fcb70 0x55cc921fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921f22b0 0x55cc921f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 --2- 172.21.15.62:0/3692110278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc921f99f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 >> 172.21.15.62:0/3692110278 conn(0x55cc921edf20 msgr2=0x55cc921f3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef757fa640 0 client.25708 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 shutdown_connections 2024-08-16T19:24:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.353+0000 7fef96188740 1 -- 172.21.15.62:0/3692110278 wait complete. 2024-08-16T19:24:48.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.354+0000 7fef96188740 20 client.25708 trim_cache size 0 max 16384 2024-08-16T19:24:48.377 INFO:tasks.workunit.client.0.smithi062.stdout: 10 12905 13.15 MB/sec warmup 66 sec latency 29302.329 ms 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.388+0000 7fef96188740 1 Processor -- start 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.388+0000 7fef96188740 1 -- start start 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.388+0000 7fef96188740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f1990 0x55cc921f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.388+0000 7fef96188740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef96188740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc92214e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef96188740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1a44c0 con 0x55cc921f1990 2024-08-16T19:24:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef96188740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1a6120 con 0x55cc921fcb70 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef96188740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1a4b90 con 0x55cc921f22b0 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:46436/0 (socket says 172.21.15.62:46436) 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 -- 172.21.15.62:0/739594303 learned_addr learned my addr 172.21.15.62:0/739594303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef92327640 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc92214e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 -- 172.21.15.62:0/739594303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 msgr2=0x55cc92214e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc92214e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 -- 172.21.15.62:0/739594303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f1990 msgr2=0x55cc921f4440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f1990 0x55cc921f4440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 -- 172.21.15.62:0/739594303 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc921e9180 con 0x55cc921f22b0 2024-08-16T19:24:48.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.389+0000 7fef91325640 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0099b0 tx=0x7fef7c033770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5800cfb65a87cf1f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.390+0000 7fef777fe640 1 -- 172.21.15.62:0/739594303 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0c2070 con 0x55cc921f22b0 2024-08-16T19:24:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.390+0000 7fef777fe640 1 -- 172.21.15.62:0/739594303 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c0c0540 con 0x55cc921f22b0 2024-08-16T19:24:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.390+0000 7fef777fe640 1 -- 172.21.15.62:0/739594303 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c04e040 con 0x55cc921f22b0 2024-08-16T19:24:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 -- 172.21.15.62:0/739594303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 msgr2=0x55cc92214900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0099b0 tx=0x7fef7c033770 comp rx=0 tx=0).stop 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 -- 172.21.15.62:0/739594303 shutdown_connections 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc92214e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f22b0 0x55cc92214900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 --2- 172.21.15.62:0/739594303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f1990 0x55cc921f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.391+0000 7fef96188740 1 -- 172.21.15.62:0/739594303 >> 172.21.15.62:0/739594303 conn(0x55cc921edf20 msgr2=0x55cc921fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.392+0000 7fef96188740 1 -- 172.21.15.62:0/739594303 shutdown_connections 2024-08-16T19:24:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.393+0000 7fef96188740 1 -- 172.21.15.62:0/739594303 wait complete. 2024-08-16T19:24:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.394+0000 7fef96188740 1 -- start start 2024-08-16T19:24:48.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.394+0000 7fef96188740 1 Processor -- start 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1a5990 con 0x55cc921f22b0 2024-08-16T19:24:48.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c1a5290 con 0x55cc921fcb70 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:46438/0 (socket says 172.21.15.62:46438) 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c073a20 con 0x55cc921f1990 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 -- 172.21.15.62:0/1838934731 learned_addr learned my addr 172.21.15.62:0/1838934731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef96188740 10 client.0 _ll_register_callbacks cb 0x7fef8800b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91325640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef92327640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 -- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 msgr2=0x55cc921f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 -- 172.21.15.62:0/1838934731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 msgr2=0x55cc921f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.395+0000 7fef91b26640 1 -- 172.21.15.62:0/1838934731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc921f7430 con 0x55cc921f1990 2024-08-16T19:24:48.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef92327640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:24:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef91b26640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef880267d0 tx=0x7fef88026800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae6ee3872a929ac server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:48.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef757fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-08-16T19:24:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88036030 con 0x55cc921f1990 2024-08-16T19:24:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fef88026c50 con 0x55cc921f1990 2024-08-16T19:24:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c0c0540 con 0x55cc921f1990 2024-08-16T19:24:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fef88040050 con 0x55cc921f1990 2024-08-16T19:24:48.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef96188740 10 client.25714 fetch_fsmap learned FSMap version 68 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.396+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55cc922800a0 con 0x55cc921f1990 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef88026e50 con 0x55cc921f1990 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 68) ==== 8402+0+0 (secure 0 0 0) 0x7fef8801ebc0 con 0x55cc921f1990 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 10 client.25714 fetch_fsmap finished waiting for FSMap version 68 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 10 client.25714 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 20 client.25714 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc92280900 0x55cc92282d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 4 client.25714 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ea9abd1c", "release": true, "await": true}] 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ea9abd1c", "release": true, "await": true}) -- 0x7fef7c0c0540 con 0x55cc92280900 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.397+0000 7fef91325640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc92280900 0x55cc92282d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.398+0000 7fef757fa640 10 client.25714 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.398+0000 7fef91325640 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc92280900 0x55cc92282d20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38197c03c1d4a200 server_cookie=4726bef7e4ed6b45 in_seq=0 out_seq=0 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.465+0000 7fef757fa640 1 -- 172.21.15.62:0/1838934731 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fef7c0c0540 con 0x55cc92280900 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.465+0000 7fef757fa640 10 client.25714 handle_command_reply: tid=0 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.465+0000 7fef96188740 1 client.25714 shutdown 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 msgr2=0x55cc922051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef880267d0 tx=0x7fef88026800 comp rx=0 tx=0).stop 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 shutdown_connections 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55cc92280900 0x55cc92282d20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc921fcb70 0x55cc921f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cc921f22b0 0x55cc921f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 --2- 172.21.15.62:0/1838934731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc921f1990 0x55cc922051c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef757fa640 0 client.25714 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 >> 172.21.15.62:0/1838934731 conn(0x55cc921edf20 msgr2=0x55cc921ec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 shutdown_connections 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 1 -- 172.21.15.62:0/1838934731 wait complete. 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.466+0000 7fef96188740 20 client.25714 trim_cache size 0 max 16384 2024-08-16T19:24:48.468 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 68, 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 52, 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "ea9abd1c": { 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 52, 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:48.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:48.470 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 msgr2=0x7fef780a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 0x7fef780a53f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef8800b2c0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 msgr2=0x7fef8c196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 0x7fef8c196570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009230 tx=0x7fef7c005230 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 shutdown_connections 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fef780a2f30 0x7fef780a53f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef8c102c20 0x7fef8c19a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef8c1022a0 0x7fef8c196570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 --2- 172.21.15.62:0/1758613826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef8c100b50 0x7fef8c196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 >> 172.21.15.62:0/1758613826 conn(0x7fef8c0fc500 msgr2=0x7fef8c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 shutdown_connections 2024-08-16T19:24:48.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:48.467+0000 7fef93d8a640 1 -- 172.21.15.62:0/1758613826 wait complete. 2024-08-16T19:24:49.206 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ea9abd1c', total seconds elapsed: 33.4 2024-08-16T19:24:49.206 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-16T19:24:49.206 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:24:49.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 13444 13.51 MB/sec warmup 67 sec latency 29422.785 ms 2024-08-16T19:24:49.564 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:49 smithi062 ceph-mon[32993]: pgmap v449: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:49 smithi161 ceph-mon[37576]: pgmap v449: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.086+0000 7f36c3af3640 1 -- 172.21.15.62:0/1286433638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc102250 msgr2=0x7f36bc10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.086+0000 7f36c3af3640 1 --2- 172.21.15.62:0/1286433638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc102250 0x7f36bc10e780 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009cd0 tx=0x7f36b402a8e0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 -- 172.21.15.62:0/1286433638 shutdown_connections 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 --2- 172.21.15.62:0/1286433638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc102250 0x7f36bc10e780 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 --2- 172.21.15.62:0/1286433638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1018b0 0x7f36bc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 --2- 172.21.15.62:0/1286433638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc1078b0 0x7f36bc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 -- 172.21.15.62:0/1286433638 >> 172.21.15.62:0/1286433638 conn(0x7f36bc0fd5f0 msgr2=0x7f36bc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 -- 172.21.15.62:0/1286433638 shutdown_connections 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 -- 172.21.15.62:0/1286433638 wait complete. 2024-08-16T19:24:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 Processor -- start 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 -- start start 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.087+0000 7f36c3af3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc102250 0x7f36bc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c3af3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1078b0 0x7f36bc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c3af3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc111cb0 con 0x7f36bc1018b0 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c3af3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc111b30 con 0x7f36bc1078b0 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c3af3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc111e30 con 0x7f36bc102250 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33032/0 (socket says 172.21.15.62:33032) 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 -- 172.21.15.62:0/503115671 learned_addr learned my addr 172.21.15.62:0/503115671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c108e640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc102250 0x7f36bc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c2090640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1078b0 0x7f36bc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 -- 172.21.15.62:0/503115671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc102250 msgr2=0x7f36bc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc102250 0x7f36bc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 -- 172.21.15.62:0/503115671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1078b0 msgr2=0x7f36bc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1078b0 0x7f36bc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 -- 172.21.15.62:0/503115671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc19d140 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c188f640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e9f0 tx=0x7f36b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800cd80 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b8010910 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc19d430 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.088+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8018990 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.089+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a4d10 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.089+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc101d10 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.091+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36b8010ac0 con 0x7f36bc1018b0 2024-08-16T19:24:50.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.091+0000 7f36b2ffd640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 0x7f36a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.092+0000 7f36c108e640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 0x7f36a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.092+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f36b80cfed0 con 0x7f36bc1018b0 2024-08-16T19:24:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.092+0000 7f36c108e640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 0x7f36a80a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ffc00 tx=0x7f36ac00b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.095+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f36b809c0b0 con 0x7f36bc1018b0 2024-08-16T19:24:50.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:49 smithi155 ceph-mon[37413]: pgmap v449: 129 pgs: 129 active+clean; 955 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.3 MiB/s wr, 149 op/s 2024-08-16T19:24:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.337+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36bc02d5e0 con 0x7f36bc1018b0 2024-08-16T19:24:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.339+0000 7f36b2ffd640 1 -- 172.21.15.62:0/503115671 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+12000 (secure 0 0 0) 0x7f36b80a03b0 con 0x7f36bc1018b0 2024-08-16T19:24:50.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:24:50.341 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"btime":"2024-08-16T19:24:09:764063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:24:09.567303+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:24:50.341 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 68 2024-08-16T19:24:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 msgr2=0x7f36a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 0x7f36a80a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ffc00 tx=0x7f36ac00b390 comp rx=0 tx=0).stop 2024-08-16T19:24:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 msgr2=0x7f36bc198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e9f0 tx=0x7f36b800eeb0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 shutdown_connections 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f36a80a2cf0 0x7f36a80a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc1078b0 0x7f36bc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36bc102250 0x7f36bc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 --2- 172.21.15.62:0/503115671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36bc1018b0 0x7f36bc198190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.342+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 >> 172.21.15.62:0/503115671 conn(0x7f36bc0fd5f0 msgr2=0x7f36bc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.343+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 shutdown_connections 2024-08-16T19:24:50.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:50.343+0000 7f36c3af3640 1 -- 172.21.15.62:0/503115671 wait complete. 2024-08-16T19:24:50.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 13851 13.68 MB/sec warmup 68 sec latency 820.377 ms 2024-08-16T19:24:50.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:50 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/503115671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:51.072 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-16T19:24:51.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-16T19:24:51.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:50 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/503115671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:51.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:50 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/503115671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:24:51.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14306 13.75 MB/sec warmup 69 sec latency 1766.659 ms 2024-08-16T19:24:51.404 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 -- 172.21.15.62:0/3895406004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc1034b0 msgr2=0x7f49bc1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/3895406004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc1034b0 0x7f49bc1078f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009930 tx=0x7f49a402a420 comp rx=0 tx=0).stop 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 -- 172.21.15.62:0/3895406004 shutdown_connections 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/3895406004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc108030 0x7f49bc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/3895406004 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc1034b0 0x7f49bc1078f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/3895406004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc102b00 0x7f49bc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 -- 172.21.15.62:0/3895406004 >> 172.21.15.62:0/3895406004 conn(0x7f49bc0fc500 msgr2=0x7f49bc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 -- 172.21.15.62:0/3895406004 shutdown_connections 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.870+0000 7f49c0d5c640 1 -- 172.21.15.62:0/3895406004 wait complete. 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 Processor -- start 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 -- start start 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc1034b0 0x7f49bc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc108030 0x7f49bc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111e10 con 0x7f49bc102b00 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111c90 con 0x7f49bc108030 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49c0d5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111f90 con 0x7f49bc1034b0 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49b9d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc1034b0 0x7f49bc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34280/0 (socket says 172.21.15.62:34280) 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.871+0000 7f49ba59c640 1 -- 172.21.15.62:0/1525713275 learned_addr learned my addr 172.21.15.62:0/1525713275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc108030 0x7f49bc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 -- 172.21.15.62:0/1525713275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc1034b0 msgr2=0x7f49bc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc1034b0 0x7f49bc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 -- 172.21.15.62:0/1525713275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc108030 msgr2=0x7f49bc19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc108030 0x7f49bc19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 -- 172.21.15.62:0/1525713275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc19d2e0 con 0x7f49bc102b00 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49ba59c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00d950 tx=0x7f49ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49b37fe640 1 -- 172.21.15.62:0/1525713275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac017030 con 0x7f49bc102b00 2024-08-16T19:24:51.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc19d5d0 con 0x7f49bc102b00 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49b37fe640 1 -- 172.21.15.62:0/1525713275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ac004020 con 0x7f49bc102b00 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49b37fe640 1 -- 172.21.15.62:0/1525713275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0047a0 con 0x7f49bc102b00 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.872+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc100690 con 0x7f49bc102b00 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.875+0000 7f49b37fe640 1 -- 172.21.15.62:0/1525713275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49ac0041d0 con 0x7f49bc102b00 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.875+0000 7f49c315a740 1 Processor -- start 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 -- start start 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c1872700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634c17caa70 con 0x5634c1865a50 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c1762780 con 0x5634c1865130 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49c315a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c17c6a30 con 0x5634c1870310 2024-08-16T19:24:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:46862/0 (socket says 172.21.15.62:46862) 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49bad9d640 1 -- 172.21.15.62:0/736315405 learned_addr learned my addr 172.21.15.62:0/736315405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49b37fe640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 0x7f49a80a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c1872700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 0x7f49a80a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.876+0000 7f49b37fe640 1 -- 172.21.15.62:0/1525713275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49ac0cf1b0 con 0x7f49bc102b00 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 -- 172.21.15.62:0/736315405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 msgr2=0x5634c1872700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c1872700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 -- 172.21.15.62:0/736315405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 msgr2=0x5634c186fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 -- 172.21.15.62:0/736315405 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c153cb90 con 0x5634c1865130 2024-08-16T19:24:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49ba59c640 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005f40 tx=0x7f49b4012f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627b50ae91f04b34 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 0x7f49a80a95c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49b401b000 tx=0x7f49b401d000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49b0ff9640 1 -- 172.21.15.62:0/736315405 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4022070 con 0x5634c1865130 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49b0ff9640 1 -- 172.21.15.62:0/736315405 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4011c40 con 0x5634c1865130 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.877+0000 7f49b0ff9640 1 -- 172.21.15.62:0/736315405 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b400f3c0 con 0x5634c1865130 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- 172.21.15.62:0/736315405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 msgr2=0x5634c1865510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005f40 tx=0x7f49b4012f00 comp rx=0 tx=0).stop 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- 172.21.15.62:0/736315405 shutdown_connections 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c1872700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 --2- 172.21.15.62:0/736315405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c1865510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- 172.21.15.62:0/736315405 >> 172.21.15.62:0/736315405 conn(0x5634c18616c0 msgr2=0x5634c1861aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:51.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- 172.21.15.62:0/736315405 shutdown_connections 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- 172.21.15.62:0/736315405 wait complete. 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 -- start start 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.878+0000 7f49c315a740 1 Processor -- start 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 0x5634c186d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 0x5634c186d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 0x5634c186dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5634c1872ee0 con 0x5634c1865130 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5634c1785990 con 0x5634c1870310 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5634c1873060 con 0x5634c1865a50 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49c315a740 10 client.0 _ll_register_callbacks cb 0x5634c14da090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 0x5634c186d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49b9d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 0x5634c186d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49b9d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 0x5634c186d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:41678/0 (socket says 172.21.15.62:41678) 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 0x5634c186dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49b9d9b640 1 -- 172.21.15.62:0/2788635191 learned_addr learned my addr 172.21.15.62:0/2788635191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 -- 172.21.15.62:0/2788635191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 msgr2=0x5634c186d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 0x5634c186d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 -- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 msgr2=0x5634c186dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 0x5634c186dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 -- 172.21.15.62:0/2788635191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c17846c0 con 0x5634c1865130 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49bad9d640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 0x5634c186dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49ba59c640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 0x5634c186d180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0cc510 tx=0x7f49ac0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ef048544685cca server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.879+0000 7f49a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac09ac40 con 0x5634c1865130 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49ac097030 con 0x5634c1865130 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b4011c40 con 0x5634c1865130 2024-08-16T19:24:51.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49ac09b9e0 con 0x5634c1865130 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 10 client.16158 fetch_fsmap learned FSMap version 69 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5634c187eb70 con 0x5634c1865130 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b4011c40 con 0x5634c1865130 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 69) ==== 8406+0+0 (secure 0 0 0) 0x7f49ac09a730 con 0x5634c1865130 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 10 client.16158 fetch_fsmap finished waiting for FSMap version 69 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 10 client.16158 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 20 client.16158 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c187ff60 0x5634c18823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 4 client.16158 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:24:51.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.880+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c17d81a0 con 0x5634c187ff60 2024-08-16T19:24:51.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.881+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c187ff60 0x5634c18823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.881+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c187ff60 0x5634c18823e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219287a22fc3c090 server_cookie=52a1029cc6e273bd in_seq=0 out_seq=0 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.881+0000 7f49a1ffb640 10 client.16158 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49a1ffb640 1 -- 172.21.15.62:0/2788635191 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634c17d81a0 con 0x5634c187ff60 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49a1ffb640 10 client.16158 handle_command_reply: tid=0 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 client.16158 shutdown 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 msgr2=0x5634c186d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 0x5634c186d180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0cc510 tx=0x7f49ac0051f0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 shutdown_connections 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c187ff60 0x5634c18823e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1870310 0x5634c186dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865a50 0x5634c186d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49a1ffb640 0 client.16158 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 --2- 172.21.15.62:0/2788635191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865130 0x5634c186d180 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.884+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 >> 172.21.15.62:0/2788635191 conn(0x5634c18616c0 msgr2=0x5634c18673f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.885+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 shutdown_connections 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.885+0000 7f49c315a740 1 -- 172.21.15.62:0/2788635191 wait complete. 2024-08-16T19:24:51.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.885+0000 7f49c315a740 20 client.16158 trim_cache size 0 max 16384 2024-08-16T19:24:51.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 Processor -- start 2024-08-16T19:24:51.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 -- start start 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865130 0x5634c186f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865a50 0x5634c1867b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1accc0 con 0x5634c1870310 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ae950 con 0x5634c1865a50 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.904+0000 7f49c315a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ad390 con 0x5634c1865130 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34312/0 (socket says 172.21.15.62:34312) 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 -- 172.21.15.62:0/1387023602 learned_addr learned my addr 172.21.15.62:0/1387023602 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:51.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865a50 0x5634c1867b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49ba59c640 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865130 0x5634c186f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 -- 172.21.15.62:0/1387023602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865130 msgr2=0x5634c186f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865130 0x5634c186f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 -- 172.21.15.62:0/1387023602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865a50 msgr2=0x5634c1867b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865a50 0x5634c1867b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 -- 172.21.15.62:0/1387023602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c1865880 con 0x5634c1870310 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49bad9d640 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f49b4012fd0 tx=0x7f49b401db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af6d651bb06576d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.905+0000 7f49a3fff640 1 -- 172.21.15.62:0/1387023602 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4022040 con 0x5634c1870310 2024-08-16T19:24:51.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49a3fff640 1 -- 172.21.15.62:0/1387023602 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b401d8e0 con 0x5634c1870310 2024-08-16T19:24:51.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49a3fff640 1 -- 172.21.15.62:0/1387023602 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011ea0 con 0x5634c1870310 2024-08-16T19:24:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 -- 172.21.15.62:0/1387023602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 msgr2=0x5634c186c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f49b4012fd0 tx=0x7f49b401db70 comp rx=0 tx=0).stop 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 -- 172.21.15.62:0/1387023602 shutdown_connections 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1870310 0x5634c186c930 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865a50 0x5634c1867b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 --2- 172.21.15.62:0/1387023602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1865130 0x5634c186f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 -- 172.21.15.62:0/1387023602 >> 172.21.15.62:0/1387023602 conn(0x5634c18616c0 msgr2=0x5634c1860710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 -- 172.21.15.62:0/1387023602 shutdown_connections 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.906+0000 7f49c315a740 1 -- 172.21.15.62:0/1387023602 wait complete. 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 -- start start 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 Processor -- start 2024-08-16T19:24:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c186ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ae1c0 con 0x5634c1865a50 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ada90 con 0x5634c1865130 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1aac90 con 0x5634c1870310 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49c315a740 10 client.0 _ll_register_callbacks cb 0x7f49ac004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49b9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:46878/0 (socket says 172.21.15.62:46878) 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.907+0000 7f49ba59c640 1 -- 172.21.15.62:0/220870890 learned_addr learned my addr 172.21.15.62:0/220870890 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49bad9d640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c186ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 -- 172.21.15.62:0/220870890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 msgr2=0x5634c186ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c186ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 -- 172.21.15.62:0/220870890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 msgr2=0x5634c186a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 -- 172.21.15.62:0/220870890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c1876700 con 0x5634c1865130 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49bad9d640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c186ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49ba59c640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0a3f70 tx=0x7f49ac097b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f633c42f285ed2e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac09ac40 con 0x5634c1865130 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49ac09f120 con 0x5634c1865130 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b401d8e0 con 0x5634c1865130 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49ac09b2e0 con 0x5634c1865130 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.908+0000 7f49c315a740 10 client.25700 fetch_fsmap learned FSMap version 69 2024-08-16T19:24:51.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5634c18eadc0 con 0x5634c1865130 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b401d8e0 con 0x5634c1865130 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 69) ==== 8406+0+0 (secure 0 0 0) 0x7f49ac0b4120 con 0x5634c1865130 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 10 client.25700 fetch_fsmap finished waiting for FSMap version 69 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 10 client.25700 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 20 client.25700 populate_metadata read hostname 'smithi062' 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c18eb610 0x5634c18eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 4 client.25700 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-16T19:24:51.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5634c17d24f0 con 0x5634c18eb610 2024-08-16T19:24:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.909+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c18eb610 0x5634c18eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:24:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.910+0000 7f49a1ffb640 10 client.25700 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:51.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:51.910+0000 7f49b9d9b640 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c18eb610 0x5634c18eda30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c7a85934de8c92 server_cookie=f656607d685beb98 in_seq=0 out_seq=0 2024-08-16T19:24:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:51 smithi161 ceph-mon[37576]: pgmap v450: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.0 MiB/s wr, 265 op/s 2024-08-16T19:24:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:51 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:51 smithi062 ceph-mon[32993]: pgmap v450: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.0 MiB/s wr, 265 op/s 2024-08-16T19:24:52.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:51 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:51 smithi155 ceph-mon[37413]: pgmap v450: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.0 MiB/s wr, 265 op/s 2024-08-16T19:24:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:51 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:52.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14614 13.80 MB/sec warmup 70 sec latency 2766.704 ms 2024-08-16T19:24:52.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49a1ffb640 1 -- 172.21.15.62:0/220870890 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5634c17d24f0 con 0x5634c18eb610 2024-08-16T19:24:52.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49a1ffb640 10 client.25700 handle_command_reply: tid=0 2024-08-16T19:24:52.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 client.25700 shutdown 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 msgr2=0x5634c186a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0a3f70 tx=0x7f49ac097b80 comp rx=0 tx=0).stop 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 shutdown_connections 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x5634c18eb610 0x5634c18eda30 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5634c1870310 0x5634c186ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49a1ffb640 0 client.25700 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634c1865a50 0x5634c186a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 --2- 172.21.15.62:0/220870890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634c1865130 0x5634c186a110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 >> 172.21.15.62:0/220870890 conn(0x5634c18616c0 msgr2=0x5634c186d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 shutdown_connections 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.509+0000 7f49c315a740 1 -- 172.21.15.62:0/220870890 wait complete. 2024-08-16T19:24:52.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c315a740 20 client.25700 trim_cache size 0 max 16384 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 69, 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 54, 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "799edd7d": { 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 54, 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:24:52.512 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "QUIESCED", 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:52.513 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:52.514 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:24:52.514 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:24:52.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 msgr2=0x7f49a80a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 0x7f49a80a95c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49b401b000 tx=0x7f49b401d000 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 msgr2=0x7f49bc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00d950 tx=0x7f49ac00de10 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 shutdown_connections 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f49a80a7100 0x7f49a80a95c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49bc108030 0x7f49bc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49bc1034b0 0x7f49bc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 --2- 172.21.15.62:0/1525713275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49bc102b00 0x7f49bc198330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 >> 172.21.15.62:0/1525713275 conn(0x7f49bc0fc500 msgr2=0x7f49bc105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 shutdown_connections 2024-08-16T19:24:52.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:24:52.510+0000 7f49c0d5c640 1 -- 172.21.15.62:0/1525713275 wait complete. 2024-08-16T19:24:53.063 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '799edd7d', quiesce took 3.9 seconds. Will release after: 26.1 2024-08-16T19:24:53.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 13.60 MB/sec warmup 71 sec latency 2859.819 ms 2024-08-16T19:24:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:53 smithi161 ceph-mon[37576]: pgmap v451: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-08-16T19:24:54.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:53 smithi062 ceph-mon[32993]: pgmap v451: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-08-16T19:24:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:53 smithi155 ceph-mon[37413]: pgmap v451: 129 pgs: 129 active+clean; 951 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-08-16T19:24:54.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 13.42 MB/sec warmup 72 sec latency 2193.367 ms 2024-08-16T19:24:55.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 13.23 MB/sec warmup 73 sec latency 3193.406 ms 2024-08-16T19:24:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:55 smithi161 ceph-mon[37576]: pgmap v452: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 329 op/s 2024-08-16T19:24:56.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:55 smithi062 ceph-mon[32993]: pgmap v452: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 329 op/s 2024-08-16T19:24:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:55 smithi155 ceph-mon[37413]: pgmap v452: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 329 op/s 2024-08-16T19:24:56.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 13.05 MB/sec warmup 74 sec latency 4193.496 ms 2024-08-16T19:24:57.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:56 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:57.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:56 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:56 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:24:57.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.88 MB/sec warmup 75 sec latency 5193.571 ms 2024-08-16T19:24:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:57 smithi161 ceph-mon[37576]: pgmap v453: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:24:58.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:57 smithi062 ceph-mon[32993]: pgmap v453: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:24:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:57 smithi155 ceph-mon[37413]: pgmap v453: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:24:58.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.71 MB/sec warmup 76 sec latency 6193.629 ms 2024-08-16T19:24:59.378 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.54 MB/sec warmup 77 sec latency 7193.682 ms 2024-08-16T19:25:00.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:24:59 smithi062 ceph-mon[32993]: pgmap v454: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:25:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:24:59 smithi155 ceph-mon[37413]: pgmap v454: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:25:00.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.38 MB/sec warmup 78 sec latency 8193.774 ms 2024-08-16T19:25:00.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:24:59 smithi161 ceph-mon[37576]: pgmap v454: 129 pgs: 129 active+clean; 970 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 255 op/s 2024-08-16T19:25:01.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.23 MB/sec warmup 79 sec latency 9193.900 ms 2024-08-16T19:25:02.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:01 smithi062 ceph-mon[32993]: pgmap v455: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2024-08-16T19:25:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:01 smithi155 ceph-mon[37413]: pgmap v455: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2024-08-16T19:25:02.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 12.07 MB/sec warmup 80 sec latency 10194.004 ms 2024-08-16T19:25:02.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:01 smithi161 ceph-mon[37576]: pgmap v455: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 276 op/s 2024-08-16T19:25:03.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.93 MB/sec warmup 81 sec latency 11194.118 ms 2024-08-16T19:25:04.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:03 smithi062 ceph-mon[32993]: pgmap v456: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-08-16T19:25:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:03 smithi155 ceph-mon[37413]: pgmap v456: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-08-16T19:25:04.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.78 MB/sec warmup 82 sec latency 12194.182 ms 2024-08-16T19:25:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:03 smithi161 ceph-mon[37576]: pgmap v456: 129 pgs: 129 active+clean; 965 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 159 op/s 2024-08-16T19:25:05.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.64 MB/sec warmup 83 sec latency 13194.306 ms 2024-08-16T19:25:06.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:05 smithi062 ceph-mon[32993]: pgmap v457: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-08-16T19:25:06.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:05 smithi155 ceph-mon[37413]: pgmap v457: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-08-16T19:25:06.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.50 MB/sec warmup 84 sec latency 14194.415 ms 2024-08-16T19:25:06.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:05 smithi161 ceph-mon[37576]: pgmap v457: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-08-16T19:25:07.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.36 MB/sec warmup 85 sec latency 15194.547 ms 2024-08-16T19:25:08.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:07 smithi062 ceph-mon[32993]: pgmap v458: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:08.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:07 smithi155 ceph-mon[37413]: pgmap v458: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:08.379 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.23 MB/sec warmup 86 sec latency 16194.656 ms 2024-08-16T19:25:08.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:07 smithi161 ceph-mon[37576]: pgmap v458: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:09.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 11.10 MB/sec warmup 87 sec latency 17194.718 ms 2024-08-16T19:25:10.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:09 smithi062 ceph-mon[32993]: pgmap v459: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:09 smithi155 ceph-mon[37413]: pgmap v459: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:10.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.98 MB/sec warmup 88 sec latency 18195.262 ms 2024-08-16T19:25:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:09 smithi161 ceph-mon[37576]: pgmap v459: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 192 KiB/s wr, 45 op/s 2024-08-16T19:25:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:10 smithi062 ceph-mon[32993]: osdmap e110: 12 total, 12 up, 12 in 2024-08-16T19:25:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:10 smithi155 ceph-mon[37413]: osdmap e110: 12 total, 12 up, 12 in 2024-08-16T19:25:11.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.85 MB/sec warmup 89 sec latency 19195.367 ms 2024-08-16T19:25:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:10 smithi161 ceph-mon[37576]: osdmap e110: 12 total, 12 up, 12 in 2024-08-16T19:25:12.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:11 smithi062 ceph-mon[32993]: pgmap v461: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:11 smithi155 ceph-mon[37413]: pgmap v461: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:12.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.73 MB/sec warmup 90 sec latency 20195.495 ms 2024-08-16T19:25:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:11 smithi161 ceph-mon[37576]: pgmap v461: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:13.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.61 MB/sec warmup 91 sec latency 21195.600 ms 2024-08-16T19:25:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:13 smithi062 ceph-mon[32993]: pgmap v462: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:13 smithi155 ceph-mon[37413]: pgmap v462: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:14.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.50 MB/sec warmup 92 sec latency 22195.699 ms 2024-08-16T19:25:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:13 smithi161 ceph-mon[37576]: pgmap v462: 129 pgs: 129 active+clean; 967 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 191 KiB/s wr, 37 op/s 2024-08-16T19:25:15.380 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.39 MB/sec warmup 93 sec latency 23195.805 ms 2024-08-16T19:25:16.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:15 smithi062 ceph-mon[32993]: pgmap v463: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:16.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:15 smithi155 ceph-mon[37413]: pgmap v463: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:16.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.28 MB/sec warmup 94 sec latency 24195.914 ms 2024-08-16T19:25:16.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:15 smithi161 ceph-mon[37576]: pgmap v463: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:17.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.17 MB/sec warmup 95 sec latency 25196.029 ms 2024-08-16T19:25:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:17 smithi161 ceph-mon[37576]: pgmap v464: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:17.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:17 smithi161 ceph-mon[37576]: osdmap e111: 12 total, 12 up, 12 in 2024-08-16T19:25:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:17 smithi062 ceph-mon[32993]: pgmap v464: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:17.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:17 smithi062 ceph-mon[32993]: osdmap e111: 12 total, 12 up, 12 in 2024-08-16T19:25:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:17 smithi155 ceph-mon[37413]: pgmap v464: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-08-16T19:25:18.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:17 smithi155 ceph-mon[37413]: osdmap e111: 12 total, 12 up, 12 in 2024-08-16T19:25:18.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 10.06 MB/sec warmup 96 sec latency 26196.160 ms 2024-08-16T19:25:19.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:25:19.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 9.96 MB/sec warmup 97 sec latency 27196.195 ms 2024-08-16T19:25:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:25:19.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:19 smithi161 ceph-mon[37576]: pgmap v466: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-08-16T19:25:19.888 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:19 smithi062 ceph-mon[32993]: pgmap v466: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.895+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/64634526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81018c0 msgr2=0x7ff8c8101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.895+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/64634526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81018c0 0x7ff8c8101d20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009930 tx=0x7ff8bc02a420 comp rx=0 tx=0).stop 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/64634526 shutdown_connections 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/64634526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 0x7ff8c810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/64634526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81018c0 0x7ff8c8101d20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/64634526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81078c0 0x7ff8c8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/64634526 >> 172.21.15.62:0/64634526 conn(0x7ff8c80fd620 msgr2=0x7ff8c80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/64634526 shutdown_connections 2024-08-16T19:25:19.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/64634526 wait complete. 2024-08-16T19:25:19.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.896+0000 7ff8cd5a4640 1 Processor -- start 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 -- start start 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81018c0 0x7ff8c8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 0x7ff8c8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8111c80 con 0x7ff8c81078c0 2024-08-16T19:25:19.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8111b00 con 0x7ff8c81018c0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8cd5a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8111e00 con 0x7ff8c8102260 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37112/0 (socket says 172.21.15.62:37112) 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 -- 172.21.15.62:0/2764026270 learned_addr learned my addr 172.21.15.62:0/2764026270 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c6ffd640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81018c0 0x7ff8c8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c67fc640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 0x7ff8c8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 -- 172.21.15.62:0/2764026270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 msgr2=0x7ff8c8074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 0x7ff8c8074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 -- 172.21.15.62:0/2764026270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81018c0 msgr2=0x7ff8c8074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81018c0 0x7ff8c8074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.897+0000 7ff8c77fe640 1 -- 172.21.15.62:0/2764026270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81a6680 con 0x7ff8c81078c0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8c77fe640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eb10 tx=0x7ff8c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:19.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0004b20 con 0x7ff8c81078c0 2024-08-16T19:25:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c0004cc0 con 0x7ff8c81078c0 2024-08-16T19:25:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81a6970 con 0x7ff8c81078c0 2024-08-16T19:25:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010760 con 0x7ff8c81078c0 2024-08-16T19:25:19.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.898+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a6e30 con 0x7ff8c81078c0 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.899+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c8101d20 con 0x7ff8c81078c0 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.900+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff8c0005020 con 0x7ff8c81078c0 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.901+0000 7ff8affff640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 0x7ff8b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.901+0000 7ff8c6ffd640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 0x7ff8b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.901+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7ff8c00cfea0 con 0x7ff8c81078c0 2024-08-16T19:25:19.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.902+0000 7ff8c6ffd640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 0x7ff8b00a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005fd0 tx=0x7ff8b8005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:19.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:19.905+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff8c009c6d0 con 0x7ff8c81078c0 2024-08-16T19:25:20.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:19 smithi155 ceph-mon[37413]: pgmap v466: 129 pgs: 129 active+clean; 964 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2024-08-16T19:25:20.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.119+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8c8109e50 con 0x7ff8c81078c0 2024-08-16T19:25:20.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.120+0000 7ff8affff640 1 -- 172.21.15.62:0/2764026270 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+12006 (secure 0 0 0) 0x7ff8c009bfd0 con 0x7ff8c81078c0 2024-08-16T19:25:20.121 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:25:20.121 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":70,"btime":"2024-08-16T19:24:55:867509+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:24:55.589088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24400,24407,24451,24469,24476]},"id":1}]} 2024-08-16T19:25:20.121 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 70 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 msgr2=0x7ff8b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 0x7ff8b00a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8005fd0 tx=0x7ff8b8005850 comp rx=0 tx=0).stop 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 msgr2=0x7ff8c8074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000eb10 tx=0x7ff8c000c2d0 comp rx=0 tx=0).stop 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 shutdown_connections 2024-08-16T19:25:20.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff8b00a2cf0 0x7ff8b00a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c81078c0 0x7ff8c8074ca0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8c8102260 0x7ff8c8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 --2- 172.21.15.62:0/2764026270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8c81018c0 0x7ff8c8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.122+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 >> 172.21.15.62:0/2764026270 conn(0x7ff8c80fd620 msgr2=0x7ff8c80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.123+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 shutdown_connections 2024-08-16T19:25:20.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:20.123+0000 7ff8cd5a4640 1 -- 172.21.15.62:0/2764026270 wait complete. 2024-08-16T19:25:20.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 9.86 MB/sec warmup 98 sec latency 28196.248 ms 2024-08-16T19:25:20.816 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 799edd7d --release --await' 2024-08-16T19:25:20.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 799edd7d --release --await 2024-08-16T19:25:20.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:20 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2764026270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:25:20.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:20 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2764026270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:25:21.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:20 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2764026270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:25:21.087 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:25:21.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 14623 9.76 MB/sec warmup 99 sec latency 29196.298 ms 2024-08-16T19:25:21.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 -- 172.21.15.62:0/1284562891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c1022a0 msgr2=0x7efd4c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1284562891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c1022a0 0x7efd4c1026e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efd40009930 tx=0x7efd4002a4d0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 -- 172.21.15.62:0/1284562891 shutdown_connections 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1284562891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c102c20 0x7efd4c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1284562891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c1022a0 0x7efd4c1026e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1284562891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c100b50 0x7efd4c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.559+0000 7efd51f3e640 1 -- 172.21.15.62:0/1284562891 >> 172.21.15.62:0/1284562891 conn(0x7efd4c0fc500 msgr2=0x7efd4c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.560+0000 7efd51f3e640 1 -- 172.21.15.62:0/1284562891 shutdown_connections 2024-08-16T19:25:21.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.560+0000 7efd51f3e640 1 -- 172.21.15.62:0/1284562891 wait complete. 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 Processor -- start 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 -- start start 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c1022a0 0x7efd4c109f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c102c20 0x7efd4c10a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10fc10 con 0x7efd4c1022a0 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10fa90 con 0x7efd4c100b50 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.561+0000 7efd51f3e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c10fd90 con 0x7efd4c102c20 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54452/0 (socket says 172.21.15.62:54452) 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 -- 172.21.15.62:0/1765617690 learned_addr learned my addr 172.21.15.62:0/1765617690 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4bfff640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c102c20 0x7efd4c10a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4affd640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c1022a0 0x7efd4c109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 -- 172.21.15.62:0/1765617690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c102c20 msgr2=0x7efd4c10a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c102c20 0x7efd4c10a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 -- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c1022a0 msgr2=0x7efd4c109f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c1022a0 0x7efd4c109f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.562+0000 7efd4b7fe640 1 -- 172.21.15.62:0/1765617690 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c073b80 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.563+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd4400ba50 tx=0x7efd4400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.564+0000 7efd48ff9640 1 -- 172.21.15.62:0/1765617690 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44019070 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.564+0000 7efd48ff9640 1 -- 172.21.15.62:0/1765617690 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4400cbe0 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.565+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c073e70 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.565+0000 7efd48ff9640 1 -- 172.21.15.62:0/1765617690 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440076e0 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.565+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c1b3c50 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.570+0000 7efd48ff9640 1 -- 172.21.15.62:0/1765617690 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd4401c070 con 0x7efd4c100b50 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.570+0000 7efd5433c740 1 Processor -- start 2024-08-16T19:25:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.570+0000 7efd5433c740 1 -- start start 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565246352c60 con 0x5652463f7b70 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5652462eab90 con 0x5652463ec990 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd5433c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56524634ec20 con 0x5652463ed2b0 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd4bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463ecd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54468/0 (socket says 172.21.15.62:54468) 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd4bfff640 1 -- 172.21.15.62:0/1941034224 learned_addr learned my addr 172.21.15.62:0/1941034224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.571+0000 7efd4affd640 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.572+0000 7efd48ff9640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 0x7efd340a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd48ff9640 1 -- 172.21.15.62:0/1765617690 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7efd440ced30 con 0x7efd4c100b50 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 -- 172.21.15.62:0/1941034224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 msgr2=0x5652463f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 -- 172.21.15.62:0/1941034224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 msgr2=0x5652463ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463ecd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 -- 172.21.15.62:0/1941034224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565246062580 con 0x5652463f7b70 2024-08-16T19:25:21.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.573+0000 7efd4affd640 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f9f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a9f0 tx=0x7efd40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958e5e2e155b283f server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.574+0000 7efd31ffb640 1 -- 172.21.15.62:0/1941034224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40035030 con 0x5652463f7b70 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.574+0000 7efd31ffb640 1 -- 172.21.15.62:0/1941034224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40004430 con 0x5652463f7b70 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.574+0000 7efd31ffb640 1 -- 172.21.15.62:0/1941034224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400035f0 con 0x5652463f7b70 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.574+0000 7efd4bfff640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 0x7efd340a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 -- 172.21.15.62:0/1941034224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 msgr2=0x5652463f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f9f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a9f0 tx=0x7efd40004040 comp rx=0 tx=0).stop 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 -- 172.21.15.62:0/1941034224 shutdown_connections 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f9f60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 --2- 172.21.15.62:0/1941034224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.575+0000 7efd5433c740 1 -- 172.21.15.62:0/1941034224 >> 172.21.15.62:0/1941034224 conn(0x5652463e8f20 msgr2=0x5652463e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.576+0000 7efd4bfff640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 0x7efd340a5380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009ee0 tx=0x7efd3c015040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.576+0000 7efd5433c740 1 -- 172.21.15.62:0/1941034224 shutdown_connections 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.576+0000 7efd5433c740 1 -- 172.21.15.62:0/1941034224 wait complete. 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.576+0000 7efd5433c740 1 -- start start 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.577+0000 7efd5433c740 1 Processor -- start 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.577+0000 7efd5433c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.577+0000 7efd5433c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.577+0000 7efd5433c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd5433c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5652463fa740 con 0x5652463f7b70 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd5433c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x5652462f07c0 con 0x5652463ec990 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd5433c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5652463fa8c0 con 0x5652463ed2b0 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47814/0 (socket says 172.21.15.62:47814) 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4affd640 1 -- 172.21.15.62:0/3518934968 learned_addr learned my addr 172.21.15.62:0/3518934968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd5433c740 10 client.0 _ll_register_callbacks cb 0x5652462f9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4bfff640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 -- 172.21.15.62:0/3518934968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 msgr2=0x5652463f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 -- 172.21.15.62:0/3518934968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 msgr2=0x5652463f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4b7fe640 1 -- 172.21.15.62:0/3518934968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524612c960 con 0x5652463ec990 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.578+0000 7efd4affd640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463f4ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd440cc140 tx=0x7efd4409a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0125725a823d2d server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd2affd640 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4409f2c0 con 0x5652463ec990 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd40004430 con 0x5652463ec990 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7efd440a36f0 con 0x5652463ec990 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7efd440a3b80 con 0x5652463ec990 2024-08-16T19:25:21.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd5433c740 10 client.25718 fetch_fsmap learned FSMap version 70 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.579+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x565246406860 con 0x5652463ec990 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40004430 con 0x5652463ec990 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 70) ==== 8414+0+0 (secure 0 0 0) 0x7efd4409a360 con 0x5652463ec990 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 10 client.25718 fetch_fsmap finished waiting for FSMap version 70 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 10 client.25718 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 20 client.25718 populate_metadata read hostname 'smithi062' 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x56524640a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 4 client.25718 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.580+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652463601d0 con 0x565246407d30 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.581+0000 7efd4affd640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x56524640a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.583+0000 7efd4affd640 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x56524640a1d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5ff64138c43769 server_cookie=5f2b390420ad3105 in_seq=0 out_seq=0 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.583+0000 7efd2affd640 10 client.25718 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:25:21.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.584+0000 7efd2affd640 1 -- 172.21.15.62:0/3518934968 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652463601d0 con 0x565246407d30 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.584+0000 7efd2affd640 10 client.25718 handle_command_reply: tid=0 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.584+0000 7efd5433c740 1 client.25718 shutdown 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 msgr2=0x5652463f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463f4ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd440cc140 tx=0x7efd4409a870 comp rx=0 tx=0).stop 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 shutdown_connections 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x56524640a1d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd2affd640 0 client.25718 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 --2- 172.21.15.62:0/3518934968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652463f4ab0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 >> 172.21.15.62:0/3518934968 conn(0x5652463e8f20 msgr2=0x5652463eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 shutdown_connections 2024-08-16T19:25:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 1 -- 172.21.15.62:0/3518934968 wait complete. 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.585+0000 7efd5433c740 20 client.25718 trim_cache size 0 max 16384 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd5433c740 1 Processor -- start 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd5433c740 1 -- start start 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd5433c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 0x5652463e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd5433c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd5433c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463f7b70 0x5652463f5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.613+0000 7efd4affd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54480/0 (socket says 172.21.15.62:54480) 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 -- 172.21.15.62:0/2698501442 learned_addr learned my addr 172.21.15.62:0/2698501442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1bdca0 con 0x5652463ec990 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1bf990 con 0x5652463ed2b0 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1be370 con 0x5652463f7b70 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4bfff640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463f7b70 0x5652463f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 0x5652463e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 -- 172.21.15.62:0/2698501442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463f7b70 msgr2=0x5652463f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463f7b70 0x5652463f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 -- 172.21.15.62:0/2698501442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 msgr2=0x5652463e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 0x5652463e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 -- 172.21.15.62:0/2698501442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565246406690 con 0x5652463ed2b0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 0x5652463e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.614+0000 7efd4affd640 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a9f0 tx=0x7efd4002a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2148e219eac7df8 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.615+0000 7efd317fa640 1 -- 172.21.15.62:0/2698501442 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40035040 con 0x5652463ed2b0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.615+0000 7efd317fa640 1 -- 172.21.15.62:0/2698501442 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400315b0 con 0x5652463ed2b0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.615+0000 7efd317fa640 1 -- 172.21.15.62:0/2698501442 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4003b690 con 0x5652463ed2b0 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 msgr2=0x5652463f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd4002a9f0 tx=0x7efd4002a4d0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 shutdown_connections 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463f7b70 0x5652463f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ed2b0 0x5652463f3a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 --2- 172.21.15.62:0/2698501442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463ec990 0x5652463e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 >> 172.21.15.62:0/2698501442 conn(0x5652463e8f20 msgr2=0x5652463f6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 shutdown_connections 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- 172.21.15.62:0/2698501442 wait complete. 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.616+0000 7efd5433c740 1 -- start start 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 Processor -- start 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652464003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1bf200 con 0x5652463f7b70 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1bea70 con 0x5652463ec990 2024-08-16T19:25:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1bbc70 con 0x5652463ed2b0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd5433c740 10 client.0 _ll_register_callbacks cb 0x7efd4409ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652464003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd4b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652464003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.62:54488/0 (socket says 172.21.15.62:54488) 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.617+0000 7efd4b7fe640 1 -- 172.21.15.62:0/2177308217 learned_addr learned my addr 172.21.15.62:0/2177308217 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4affd640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 -- 172.21.15.62:0/2177308217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 msgr2=0x5652463f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 -- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 msgr2=0x5652464003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652464003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.618+0000 7efd4bfff640 1 -- 172.21.15.62:0/2177308217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565246471ab0 con 0x5652463f7b70 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.619+0000 7efd4bfff640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f2150 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009a50 tx=0x7efd3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fede97c7947827 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:25:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.619+0000 7efd2affd640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.619+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02e060 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.619+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7efd3c013660 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.619+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd400315b0 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.620+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7efd3c035050 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.620+0000 7efd5433c740 10 client.16185 fetch_fsmap learned FSMap version 70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.620+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x565246472d10 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.620+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400315b0 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 70) ==== 8414+0+0 (secure 0 0 0) 0x7efd3c0298c0 con 0x5652463f7b70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 10 client.16185 fetch_fsmap finished waiting for FSMap version 70 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 10 client.16185 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 20 client.16185 populate_metadata read hostname 'smithi062' 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x565246475540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 4 client.16185 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "799edd7d", "release": true, "await": true}] 2024-08-16T19:25:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.621+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "set_id": "799edd7d", "release": true, "await": true}) -- 0x56524635a6e0 con 0x565246407d30 2024-08-16T19:25:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.623+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x565246475540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:25:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.623+0000 7efd4b7fe640 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x565246475540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425b194ecc49c6da server_cookie=31822f956dfbf7cc in_seq=0 out_seq=0 2024-08-16T19:25:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.623+0000 7efd2affd640 10 client.16185 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:25:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.680+0000 7efd2affd640 1 -- 172.21.15.62:0/2177308217 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56524635a6e0 con 0x565246407d30 2024-08-16T19:25:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd2affd640 10 client.16185 handle_command_reply: tid=0 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 client.16185 shutdown 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 msgr2=0x5652463f2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f2150 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009a50 tx=0x7efd3c0046c0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 shutdown_connections 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x565246407d30 0x565246475540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5652463f7b70 0x5652463f2150 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5652463ed2b0 0x5652463f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 --2- 172.21.15.62:0/2177308217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652463ec990 0x5652464003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd2affd640 0 client.16185 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 >> 172.21.15.62:0/2177308217 conn(0x5652463e8f20 msgr2=0x5652463f6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 shutdown_connections 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 1 -- 172.21.15.62:0/2177308217 wait complete. 2024-08-16T19:25:21.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.681+0000 7efd5433c740 20 client.16185 trim_cache size 0 max 16384 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 70, 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 56, 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": { 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "799edd7d": { 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "version": 56, 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "age_ref": 0.0, 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:25:21.684 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "timeout": 90.0, 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "expiration": 150.0, 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "members": { 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "file:/": { 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "excluded": false, 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "state": { 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RELEASED", 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: "age": 0.0 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:25:21.685 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 msgr2=0x7efd340a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 0x7efd340a5380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009ee0 tx=0x7efd3c015040 comp rx=0 tx=0).stop 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 msgr2=0x7efd4c1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd4400ba50 tx=0x7efd4400bf10 comp rx=0 tx=0).stop 2024-08-16T19:25:21.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 shutdown_connections 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efd340a2ec0 0x7efd340a5380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd4c102c20 0x7efd4c10a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd4c1022a0 0x7efd4c109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 --2- 172.21.15.62:0/1765617690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c100b50 0x7efd4c1099d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 >> 172.21.15.62:0/1765617690 conn(0x7efd4c0fc500 msgr2=0x7efd4c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 shutdown_connections 2024-08-16T19:25:21.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:25:21.682+0000 7efd51f3e640 1 -- 172.21.15.62:0/1765617690 wait complete. 2024-08-16T19:25:21.746 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:21 smithi062 ceph-mon[32993]: pgmap v467: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:21.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:21 smithi161 ceph-mon[37576]: pgmap v467: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:22.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:21 smithi155 ceph-mon[37413]: pgmap v467: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:22.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 15016 9.81 MB/sec warmup 100 sec latency 29497.292 ms 2024-08-16T19:25:22.595 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '799edd7d', total seconds elapsed: 33.4 2024-08-16T19:25:23.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 15576 10.09 MB/sec warmup 101 sec latency 56.055 ms 2024-08-16T19:25:23.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:23 smithi062 ceph-mon[32993]: pgmap v468: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:24.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:23 smithi155 ceph-mon[37413]: pgmap v468: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:24.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:23 smithi161 ceph-mon[37576]: pgmap v468: 129 pgs: 129 active+clean; 974 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-08-16T19:25:24.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 16196 10.21 MB/sec warmup 102 sec latency 54.672 ms 2024-08-16T19:25:25.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 16862 10.49 MB/sec warmup 103 sec latency 742.932 ms 2024-08-16T19:25:25.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:25 smithi062 ceph-mon[32993]: pgmap v469: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:26.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:25 smithi155 ceph-mon[37413]: pgmap v469: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:26.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:25 smithi161 ceph-mon[37576]: pgmap v469: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:26.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 17442 10.77 MB/sec warmup 104 sec latency 1742.999 ms 2024-08-16T19:25:27.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 17780 10.88 MB/sec warmup 105 sec latency 2743.033 ms 2024-08-16T19:25:27.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:27 smithi062 ceph-mon[32993]: pgmap v470: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:28.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:27 smithi155 ceph-mon[37413]: pgmap v470: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:28.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:27 smithi161 ceph-mon[37576]: pgmap v470: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 220 op/s 2024-08-16T19:25:28.381 INFO:tasks.workunit.client.0.smithi062.stdout: 10 18271 10.95 MB/sec warmup 106 sec latency 3743.068 ms 2024-08-16T19:25:29.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 18691 11.06 MB/sec warmup 107 sec latency 4743.103 ms 2024-08-16T19:25:29.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:29 smithi062 ceph-mon[32993]: pgmap v471: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-08-16T19:25:30.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:29 smithi155 ceph-mon[37413]: pgmap v471: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-08-16T19:25:30.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:29 smithi161 ceph-mon[37576]: pgmap v471: 129 pgs: 129 active+clean; 986 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.5 MiB/s wr, 185 op/s 2024-08-16T19:25:30.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 19172 11.22 MB/sec warmup 108 sec latency 5743.429 ms 2024-08-16T19:25:31.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 19830 11.34 MB/sec warmup 109 sec latency 6743.459 ms 2024-08-16T19:25:31.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:31 smithi062 ceph-mon[32993]: pgmap v472: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-08-16T19:25:32.078 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:31 smithi155 ceph-mon[37413]: pgmap v472: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-08-16T19:25:32.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:31 smithi161 ceph-mon[37576]: pgmap v472: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-08-16T19:25:32.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 20365 11.60 MB/sec warmup 110 sec latency 7743.500 ms 2024-08-16T19:25:33.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 20839 11.74 MB/sec warmup 111 sec latency 8743.547 ms 2024-08-16T19:25:33.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:33 smithi062 ceph-mon[32993]: pgmap v473: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-08-16T19:25:34.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:33 smithi155 ceph-mon[37413]: pgmap v473: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-08-16T19:25:34.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:33 smithi161 ceph-mon[37576]: pgmap v473: 129 pgs: 129 active+clean; 1010 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 306 op/s 2024-08-16T19:25:34.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 21286 11.82 MB/sec warmup 112 sec latency 9743.612 ms 2024-08-16T19:25:35.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 21642 11.87 MB/sec warmup 113 sec latency 10743.641 ms 2024-08-16T19:25:35.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:35 smithi062 ceph-mon[32993]: pgmap v474: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 445 op/s 2024-08-16T19:25:36.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:35 smithi155 ceph-mon[37413]: pgmap v474: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 445 op/s 2024-08-16T19:25:36.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:35 smithi161 ceph-mon[37576]: pgmap v474: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.7 MiB/s wr, 445 op/s 2024-08-16T19:25:36.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 22066 11.99 MB/sec warmup 114 sec latency 11130.963 ms 2024-08-16T19:25:37.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 22746 12.09 MB/sec warmup 115 sec latency 2786.975 ms 2024-08-16T19:25:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:37 smithi062 ceph-mon[32993]: pgmap v475: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:37.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:37 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:37 smithi155 ceph-mon[37413]: pgmap v475: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:38.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:37 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:37 smithi161 ceph-mon[37576]: pgmap v475: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:37 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:38.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 23522 12.42 MB/sec warmup 116 sec latency 3787.005 ms 2024-08-16T19:25:39.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 24168 12.67 MB/sec warmup 117 sec latency 4787.064 ms 2024-08-16T19:25:39.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:39 smithi062 ceph-mon[32993]: pgmap v476: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:40.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:39 smithi155 ceph-mon[37413]: pgmap v476: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:40.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:39 smithi161 ceph-mon[37576]: pgmap v476: 129 pgs: 129 active+clean; 995 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2024-08-16T19:25:40.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 24667 12.74 MB/sec warmup 118 sec latency 5243.714 ms 2024-08-16T19:25:41.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 25556 13.05 MB/sec warmup 119 sec latency 57.113 ms 2024-08-16T19:25:41.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:41 smithi062 ceph-mon[32993]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 496 op/s 2024-08-16T19:25:42.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:41 smithi155 ceph-mon[37413]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 496 op/s 2024-08-16T19:25:42.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:41 smithi161 ceph-mon[37576]: pgmap v477: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 496 op/s 2024-08-16T19:25:43.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 27390 42.96 MB/sec execute 1 sec latency 54.379 ms 2024-08-16T19:25:43.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:43 smithi062 ceph-mon[32993]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-08-16T19:25:44.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:43 smithi155 ceph-mon[37413]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-08-16T19:25:44.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:43 smithi161 ceph-mon[37576]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 337 op/s 2024-08-16T19:25:44.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 28090 37.02 MB/sec execute 2 sec latency 82.134 ms 2024-08-16T19:25:44.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:44 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:45.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:44 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:45.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:44 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:45.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 28982 42.17 MB/sec execute 3 sec latency 66.190 ms 2024-08-16T19:25:45.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:45 smithi062 ceph-mon[32993]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.6 MiB/s wr, 614 op/s 2024-08-16T19:25:46.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:45 smithi155 ceph-mon[37413]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.6 MiB/s wr, 614 op/s 2024-08-16T19:25:46.142 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:45 smithi161 ceph-mon[37576]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.6 MiB/s wr, 614 op/s 2024-08-16T19:25:46.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 29858 44.42 MB/sec execute 4 sec latency 90.221 ms 2024-08-16T19:25:46.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:46 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:25:47.039 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:46 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:25:47.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:46 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:25:47.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 30425 42.29 MB/sec execute 5 sec latency 139.613 ms 2024-08-16T19:25:47.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:47 smithi062 ceph-mon[32993]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:47 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:47 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:25:47.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:47 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:25:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:47 smithi155 ceph-mon[37413]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:47 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:25:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:47 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:25:48.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:47 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:25:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:47 smithi161 ceph-mon[37576]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:47 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:25:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:47 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:25:48.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:47 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:25:48.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 30509 36.00 MB/sec execute 6 sec latency 639.847 ms 2024-08-16T19:25:49.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 30763 31.90 MB/sec execute 7 sec latency 436.781 ms 2024-08-16T19:25:49.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:49 smithi062 ceph-mon[32993]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:50.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:49 smithi155 ceph-mon[37413]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:50.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:49 smithi161 ceph-mon[37576]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 475 op/s 2024-08-16T19:25:50.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 31274 30.48 MB/sec execute 8 sec latency 81.486 ms 2024-08-16T19:25:51.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 31988 31.97 MB/sec execute 9 sec latency 90.881 ms 2024-08-16T19:25:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:51 smithi062 ceph-mon[32993]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-16T19:25:51.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:51 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:52.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:51 smithi155 ceph-mon[37413]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-16T19:25:52.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:51 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:51 smithi161 ceph-mon[37576]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-16T19:25:52.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:51 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:25:52.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 33003 34.49 MB/sec execute 10 sec latency 73.500 ms 2024-08-16T19:25:53.382 INFO:tasks.workunit.client.0.smithi062.stdout: 10 33574 34.72 MB/sec execute 11 sec latency 204.065 ms 2024-08-16T19:25:53.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:53 smithi062 ceph-mon[32993]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-08-16T19:25:54.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:53 smithi155 ceph-mon[37413]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-08-16T19:25:54.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:53 smithi161 ceph-mon[37576]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 711 op/s 2024-08-16T19:25:54.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 34096 33.47 MB/sec execute 12 sec latency 104.142 ms 2024-08-16T19:25:55.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 34734 33.08 MB/sec execute 13 sec latency 151.702 ms 2024-08-16T19:25:55.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:55 smithi062 ceph-mon[32993]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-08-16T19:25:56.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:55 smithi155 ceph-mon[37413]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-08-16T19:25:56.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:55 smithi161 ceph-mon[37576]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-08-16T19:25:56.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 35714 34.48 MB/sec execute 14 sec latency 44.037 ms 2024-08-16T19:25:57.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 36343 34.80 MB/sec execute 15 sec latency 265.102 ms 2024-08-16T19:25:57.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:57 smithi062 ceph-mon[32993]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:25:58.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:57 smithi155 ceph-mon[37413]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:25:58.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:57 smithi161 ceph-mon[37576]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:25:58.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 37074 35.76 MB/sec execute 16 sec latency 74.848 ms 2024-08-16T19:25:59.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 37665 35.03 MB/sec execute 17 sec latency 60.663 ms 2024-08-16T19:25:59.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:25:59 smithi062 ceph-mon[32993]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:26:00.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:25:59 smithi155 ceph-mon[37413]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:26:00.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:25:59 smithi161 ceph-mon[37576]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-08-16T19:26:00.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 38197 34.39 MB/sec execute 18 sec latency 328.481 ms 2024-08-16T19:26:01.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 38585 33.23 MB/sec execute 19 sec latency 263.062 ms 2024-08-16T19:26:01.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:01 smithi161 ceph-mon[37576]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-08-16T19:26:01.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:01 smithi062 ceph-mon[32993]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-08-16T19:26:02.079 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:01 smithi155 ceph-mon[37413]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-08-16T19:26:02.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 38598 31.74 MB/sec execute 20 sec latency 429.783 ms 2024-08-16T19:26:02.996 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:02 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:03.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:02 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:03.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 38761 30.66 MB/sec execute 21 sec latency 201.942 ms 2024-08-16T19:26:03.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:02 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:04.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:03 smithi062 ceph-mon[32993]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2024-08-16T19:26:04.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:03 smithi155 ceph-mon[37413]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2024-08-16T19:26:04.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 39775 32.06 MB/sec execute 22 sec latency 158.838 ms 2024-08-16T19:26:04.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:03 smithi161 ceph-mon[37576]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 629 op/s 2024-08-16T19:26:05.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 40564 32.89 MB/sec execute 23 sec latency 112.650 ms 2024-08-16T19:26:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:05 smithi062 ceph-mon[32993]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.8 MiB/s wr, 986 op/s 2024-08-16T19:26:06.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:05 smithi062 ceph-mon[32993]: osdmap e112: 12 total, 12 up, 12 in 2024-08-16T19:26:06.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:05 smithi155 ceph-mon[37413]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.8 MiB/s wr, 986 op/s 2024-08-16T19:26:06.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:05 smithi155 ceph-mon[37413]: osdmap e112: 12 total, 12 up, 12 in 2024-08-16T19:26:06.383 INFO:tasks.workunit.client.0.smithi062.stdout: 10 41088 32.66 MB/sec execute 24 sec latency 72.485 ms 2024-08-16T19:26:06.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:05 smithi161 ceph-mon[37576]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.8 MiB/s wr, 986 op/s 2024-08-16T19:26:06.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:05 smithi161 ceph-mon[37576]: osdmap e112: 12 total, 12 up, 12 in 2024-08-16T19:26:06.572 INFO:tasks.workunit.client.0.smithi062.stdout:[44122] write failed on handle 10843 (Resource temporarily unavailable) 2024-08-16T19:26:06.572 INFO:tasks.workunit.client.0.smithi062.stdout:Child failed with status 1 2024-08-16T19:26:06.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:26:06.583 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-08-16T19:26:06.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-16T19:26:06.893 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=998c2f980771f4723d937d5cd7891f92dd116699 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-16T19:26:08.399 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c0ddac9668af42008874718194cf6cb9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_998c2f980771f4723d937d5cd7891f92dd116699/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_4e29fccbc48636195c0aac083d870323c5832411/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=998c2f980771f4723d937d5cd7891f92dd116699 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-16T19:26:08.401 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-16T19:26:09.483 INFO:tasks.quiescer:joining Quiescers 2024-08-16T19:26:09.483 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-16T19:26:09.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:26:09.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:06 smithi161 ceph-mon[37576]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:09.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:07 smithi161 ceph-mon[37576]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:09.490 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:06 smithi155 ceph-mon[37413]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:09.490 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:07 smithi155 ceph-mon[37413]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:09.490 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:06 smithi062 ceph-mon[32993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-16T19:26:09.491 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:07 smithi062 ceph-mon[32993]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:09.769 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:10.216 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:09 smithi062 ceph-mon[32993]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:10.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:09 smithi155 ceph-mon[37413]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:10.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:09 smithi161 ceph-mon[37576]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 740 op/s 2024-08-16T19:26:10.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 -- 172.21.15.62:0/1117892629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 msgr2=0x7ffb6410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:10.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 --2- 172.21.15.62:0/1117892629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb6410b8e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009900 tx=0x7ffb4c02a480 comp rx=0 tx=0).stop 2024-08-16T19:26:10.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 -- 172.21.15.62:0/1117892629 shutdown_connections 2024-08-16T19:26:10.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 --2- 172.21.15.62:0/1117892629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6410beb0 0x7ffb6410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 --2- 172.21.15.62:0/1117892629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb6410b8e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 --2- 172.21.15.62:0/1117892629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb64102ad0 0x7ffb64102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 -- 172.21.15.62:0/1117892629 >> 172.21.15.62:0/1117892629 conn(0x7ffb640fc4d0 msgr2=0x7ffb640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 -- 172.21.15.62:0/1117892629 shutdown_connections 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.300+0000 7ffb68a63640 1 -- 172.21.15.62:0/1117892629 wait complete. 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 Processor -- start 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 -- start start 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64102ad0 0x7ffb64100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:10.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6410beb0 0x7ffb641011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64111cd0 con 0x7ffb641033f0 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64111b50 con 0x7ffb64102ad0 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb68a63640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64111e50 con 0x7ffb6410beb0 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb59d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48502/0 (socket says 172.21.15.62:48502) 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb59d9b640 1 -- 172.21.15.62:0/3444106580 learned_addr learned my addr 172.21.15.62:0/3444106580 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:10.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb6259c640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64102ad0 0x7ffb64100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.301+0000 7ffb62d9d640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6410beb0 0x7ffb641011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 -- 172.21.15.62:0/3444106580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6410beb0 msgr2=0x7ffb641011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6410beb0 0x7ffb641011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 -- 172.21.15.62:0/3444106580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64102ad0 msgr2=0x7ffb64100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64102ad0 0x7ffb64100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 -- 172.21.15.62:0/3444106580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641a66a0 con 0x7ffb641033f0 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb59d9b640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006010 tx=0x7ffb4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:10.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c03f030 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c003ce0 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641a6930 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035070 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.302+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641a6e10 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.303+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb64102eb0 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.304+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb4c042030 con 0x7ffb641033f0 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.305+0000 7ffb5bfff640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 0x7ffb500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:10.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.305+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9613+0+0 (secure 0 0 0) 0x7ffb4c0f0000 con 0x7ffb641033f0 2024-08-16T19:26:10.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.305+0000 7ffb6259c640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 0x7ffb500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:10.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.305+0000 7ffb6259c640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 0x7ffb500a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005850 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:10.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.309+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffb4c0bc8a0 con 0x7ffb641033f0 2024-08-16T19:26:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.563+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb64106ef0 con 0x7ffb641033f0 2024-08-16T19:26:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.564+0000 7ffb5bfff640 1 -- 172.21.15.62:0/3444106580 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+12010 (secure 0 0 0) 0x7ffb4c0bc1a0 con 0x7ffb641033f0 2024-08-16T19:26:10.565 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:10.565 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"btime":"2024-08-16T19:26:05:936217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:26:05.891993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:26:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 75 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 msgr2=0x7ffb500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 0x7ffb500a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005fd0 tx=0x7ffb54005850 comp rx=0 tx=0).stop 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 msgr2=0x7ffb64100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006010 tx=0x7ffb4c0042f0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 shutdown_connections 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ffb500a2cf0 0x7ffb500a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb6410beb0 0x7ffb641011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb641033f0 0x7ffb64100c70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 --2- 172.21.15.62:0/3444106580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64102ad0 0x7ffb64100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 >> 172.21.15.62:0/3444106580 conn(0x7ffb640fc4d0 msgr2=0x7ffb640fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 shutdown_connections 2024-08-16T19:26:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:10.566+0000 7ffb68a63640 1 -- 172.21.15.62:0/3444106580 wait complete. 2024-08-16T19:26:11.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:10 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3444106580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:11.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:10 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3444106580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:11.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --cancel --all' 2024-08-16T19:26:11.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --cancel --all 2024-08-16T19:26:11.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:10 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3444106580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:11.644 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:12.034 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:11 smithi062 ceph-mon[32993]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-08-16T19:26:12.035 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:11 smithi062 ceph-mon[32993]: osdmap e113: 12 total, 12 up, 12 in 2024-08-16T19:26:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 -- 172.21.15.62:0/1240169838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601078b0 msgr2=0x7f7160107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 --2- 172.21.15.62:0/1240169838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601078b0 0x7f7160107c90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f715c009930 tx=0x7f715c02a420 comp rx=0 tx=0).stop 2024-08-16T19:26:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 -- 172.21.15.62:0/1240169838 shutdown_connections 2024-08-16T19:26:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 --2- 172.21.15.62:0/1240169838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7160102250 0x7f716010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 --2- 172.21.15.62:0/1240169838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71601018b0 0x7f7160101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 --2- 172.21.15.62:0/1240169838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601078b0 0x7f7160107c90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 -- 172.21.15.62:0/1240169838 >> 172.21.15.62:0/1240169838 conn(0x7f71600fd5f0 msgr2=0x7f71600ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 -- 172.21.15.62:0/1240169838 shutdown_connections 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.121+0000 7f7167695640 1 -- 172.21.15.62:0/1240169838 wait complete. 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 Processor -- start 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 -- start start 2024-08-16T19:26:12.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7160102250 0x7f71601986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71601078b0 0x7f716019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7160111c70 con 0x7f71601018b0 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7160111af0 con 0x7f7160102250 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7167695640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7160111df0 con 0x7f71601078b0 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50562/0 (socket says 172.21.15.62:50562) 2024-08-16T19:26:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7165431640 1 -- 172.21.15.62:0/3108746652 learned_addr learned my addr 172.21.15.62:0/3108746652 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.122+0000 7f7165c32640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71601078b0 0x7f716019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 -- 172.21.15.62:0/3108746652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71601078b0 msgr2=0x7f716019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71601078b0 0x7f716019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 -- 172.21.15.62:0/3108746652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7160102250 msgr2=0x7f71601986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7164c30640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7160102250 0x7f71601986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7160102250 0x7f71601986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 -- 172.21.15.62:0/3108746652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716019d160 con 0x7f71601018b0 2024-08-16T19:26:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7165431640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f715c009230 tx=0x7f715c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f71567fc640 1 -- 172.21.15.62:0/3108746652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c03f030 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716019d3f0 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f71567fc640 1 -- 172.21.15.62:0/3108746652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c003ad0 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f71567fc640 1 -- 172.21.15.62:0/3108746652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c035070 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.123+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a4d30 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.125+0000 7f71567fc640 1 -- 172.21.15.62:0/3108746652 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f715c042030 con 0x7f71601018b0 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 Processor -- start 2024-08-16T19:26:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 -- start start 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e584a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e591c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4e58c0 con 0x55b30e584fc0 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4979e0 con 0x55b30e58f880 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7169a93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4f3540 con 0x55b30e5846a0 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50576/0 (socket says 172.21.15.62:50576) 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7165431640 1 -- 172.21.15.62:0/929334996 learned_addr learned my addr 172.21.15.62:0/929334996 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7165c32640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e584a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f71567fc640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 0x7f714c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.126+0000 7f7164c30640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e591c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 0x7f714c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 -- 172.21.15.62:0/929334996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 msgr2=0x55b30e591c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f71567fc640 1 -- 172.21.15.62:0/3108746652 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f715c0efd50 con 0x7f71601018b0 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e591c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 -- 172.21.15.62:0/929334996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 msgr2=0x55b30e58f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 -- 172.21.15.62:0/929334996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30e481640 con 0x55b30e5846a0 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165431640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e584a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71580149a0 tx=0x7f7158014e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0756f1d19d0b793 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f7165c32640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 0x7f714c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f715801c000 tx=0x7f715801e000 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f714b7fe640 1 -- 172.21.15.62:0/929334996 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158014030 con 0x55b30e5846a0 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f714b7fe640 1 -- 172.21.15.62:0/929334996 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158023070 con 0x55b30e5846a0 2024-08-16T19:26:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.127+0000 7f714b7fe640 1 -- 172.21.15.62:0/929334996 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580138e0 con 0x55b30e5846a0 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- 172.21.15.62:0/929334996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 msgr2=0x55b30e584a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e584a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71580149a0 tx=0x7f7158014e60 comp rx=0 tx=0).stop 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- 172.21.15.62:0/929334996 shutdown_connections 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e591c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 --2- 172.21.15.62:0/929334996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e584a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- 172.21.15.62:0/929334996 >> 172.21.15.62:0/929334996 conn(0x55b30e580c30 msgr2=0x55b30e581010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- 172.21.15.62:0/929334996 shutdown_connections 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- 172.21.15.62:0/929334996 wait complete. 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.128+0000 7f7169a93740 1 -- start start 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 Processor -- start 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e58c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4e8c10 con 0x55b30e584fc0 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4a7f00 con 0x55b30e58f880 2024-08-16T19:26:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b30e4edf30 con 0x55b30e5846a0 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7169a93740 10 client.0 _ll_register_callbacks cb 0x55b30e1f8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50590/0 (socket says 172.21.15.62:50590) 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 -- 172.21.15.62:0/2320092278 learned_addr learned my addr 172.21.15.62:0/2320092278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7165431640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7165c32640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e58c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 -- 172.21.15.62:0/2320092278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 msgr2=0x55b30e58f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 -- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 msgr2=0x55b30e58c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e58c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7164c30640 1 -- 172.21.15.62:0/2320092278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30e25b620 con 0x55b30e584fc0 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.129+0000 7f7165431640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7164c30640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7150009d40 tx=0x7f71500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff64e12c726acee server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150012860 con 0x55b30e584fc0 2024-08-16T19:26:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f7150012a00 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7158023070 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7150025050 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7169a93740 10 client.16203 fetch_fsmap learned FSMap version 75 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55b30e59df10 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158023070 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.130+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 75) ==== 8418+0+0 (secure 0 0 0) 0x7f715000f040 con 0x55b30e584fc0 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 10 client.16203 fetch_fsmap finished waiting for FSMap version 75 2024-08-16T19:26:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 10 client.16203 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 20 client.16203 populate_metadata read hostname 'smithi062' 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e59f4c0 0x55b30e5a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 4 client.16203 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b30e4f6e70 con 0x55b30e59f4c0 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7165431640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e59f4c0 0x55b30e5a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7165431640 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e59f4c0 0x55b30e5a1940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d712b9c5407d30b1 server_cookie=4951a33bf8223526 in_seq=0 out_seq=0 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.131+0000 7f7148ff9640 10 client.16203 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7148ff9640 1 -- 172.21.15.62:0/2320092278 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b30e4f6e70 con 0x55b30e59f4c0 2024-08-16T19:26:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7148ff9640 10 client.16203 handle_command_reply: tid=0 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 client.16203 shutdown 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 msgr2=0x55b30e58c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7150009d40 tx=0x7f71500117b0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 shutdown_connections 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e59f4c0 0x55b30e5a1940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e58c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e584fc0 0x55b30e58c480 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 --2- 172.21.15.62:0/2320092278 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7148ff9640 0 client.16203 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:26:12.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.132+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 >> 172.21.15.62:0/2320092278 conn(0x55b30e580c30 msgr2=0x55b30e586960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.133+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 shutdown_connections 2024-08-16T19:26:12.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.133+0000 7f7169a93740 1 -- 172.21.15.62:0/2320092278 wait complete. 2024-08-16T19:26:12.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.133+0000 7f7169a93740 20 client.16203 trim_cache size 0 max 16384 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 Processor -- start 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 -- start start 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 0x55b30e5a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b0cd0 con 0x55b30e58f880 2024-08-16T19:26:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71601af6e0 con 0x55b30e584fc0 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.150+0000 7f7169a93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71601afdb0 con 0x55b30e5846a0 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50606/0 (socket says 172.21.15.62:50606) 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 -- 172.21.15.62:0/1073436790 learned_addr learned my addr 172.21.15.62:0/1073436790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165431640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7164c30640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 0x55b30e5a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 -- 172.21.15.62:0/1073436790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 msgr2=0x55b30e58e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 -- 172.21.15.62:0/1073436790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 msgr2=0x55b30e5a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 0x55b30e5a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 -- 172.21.15.62:0/1073436790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30e581da0 con 0x55b30e58f880 2024-08-16T19:26:12.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7164c30640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 0x55b30e5a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:26:12.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f7165c32640 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7158022000 tx=0x7f7158009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbc1246e1b84daf server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f714affd640 1 -- 172.21.15.62:0/1073436790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158012c90 con 0x55b30e58f880 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.151+0000 7f714affd640 1 -- 172.21.15.62:0/1073436790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158023050 con 0x55b30e58f880 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f714affd640 1 -- 172.21.15.62:0/1073436790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158014290 con 0x55b30e58f880 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 -- 172.21.15.62:0/1073436790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 msgr2=0x55b30e5a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7158022000 tx=0x7f7158009600 comp rx=0 tx=0).stop 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 -- 172.21.15.62:0/1073436790 shutdown_connections 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e58f880 0x55b30e5a7750 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e584fc0 0x55b30e5a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 --2- 172.21.15.62:0/1073436790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e5846a0 0x55b30e58e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 -- 172.21.15.62:0/1073436790 >> 172.21.15.62:0/1073436790 conn(0x55b30e580c30 msgr2=0x55b30e58b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 -- 172.21.15.62:0/1073436790 shutdown_connections 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.152+0000 7f7169a93740 1 -- 172.21.15.62:0/1073436790 wait complete. 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 -- start start 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 Processor -- start 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 0x55b30e595a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e595fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f71601acf50 con 0x55b30e5846a0 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b0540 con 0x55b30e58f880 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f71601ad5a0 con 0x55b30e584fc0 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7169a93740 10 client.0 _ll_register_callbacks cb 0x7f7150009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7164c30640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 0x55b30e595a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7165431640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50608/0 (socket says 172.21.15.62:50608) 2024-08-16T19:26:12.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7165431640 1 -- 172.21.15.62:0/493647201 learned_addr learned my addr 172.21.15.62:0/493647201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.153+0000 7f7164c30640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 0x55b30e595a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.62:47748/0 (socket says 172.21.15.62:47748) 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165c32640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e595fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 -- 172.21.15.62:0/493647201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 msgr2=0x55b30e595a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 0x55b30e595a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 -- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 msgr2=0x55b30e595fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e595fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 -- 172.21.15.62:0/493647201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30e609490 con 0x55b30e5846a0 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165c32640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e595fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:12.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7165431640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f715c0ed6b0 tx=0x7f715c0f2260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e537f479e461a94 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7148ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-08-16T19:26:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c049040 con 0x55b30e5846a0 2024-08-16T19:26:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f715c003c80 con 0x55b30e5846a0 2024-08-16T19:26:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7158023050 con 0x55b30e5846a0 2024-08-16T19:26:12.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f715c0bc580 con 0x55b30e5846a0 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7169a93740 10 client.16209 fetch_fsmap learned FSMap version 75 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55b30e60a770 con 0x55b30e5846a0 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.154+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7158023050 con 0x55b30e5846a0 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 75) ==== 8418+0+0 (secure 0 0 0) 0x7f715c0ed330 con 0x55b30e5846a0 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 10 client.16209 fetch_fsmap finished waiting for FSMap version 75 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 10 client.16209 resolve_mds: validated gid 24476 aka daemon mds.b 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 20 client.16209 populate_metadata read hostname 'smithi062' 2024-08-16T19:26:12.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e60ae70 0x55b30e60d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 4 client.16209 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 --> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55b30e4f1380 con 0x55b30e60ae70 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.155+0000 7f7164c30640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e60ae70 0x55b30e60d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7148ff9640 10 client.16209 ms_handle_connect on v2:172.21.15.155:6838/2045382457 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7164c30640 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e60ae70 0x55b30e60d2b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59445471c4f8211e server_cookie=a89977b1449ee64b in_seq=0 out_seq=0 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7148ff9640 1 -- 172.21.15.62:0/493647201 <== mds.0 v2:172.21.15.155:6838/2045382457 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55b30e4f1380 con 0x55b30e60ae70 2024-08-16T19:26:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7148ff9640 10 client.16209 handle_command_reply: tid=0 2024-08-16T19:26:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 client.16209 shutdown 2024-08-16T19:26:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 msgr2=0x55b30e595550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f715c0ed6b0 tx=0x7f715c0f2260 comp rx=0 tx=0).stop 2024-08-16T19:26:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 shutdown_connections 2024-08-16T19:26:12.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:6838/2045382457,v1:172.21.15.155:6839/2045382457] conn(0x55b30e60ae70 0x55b30e60d2b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b30e58f880 0x55b30e595fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b30e584fc0 0x55b30e595a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7148ff9640 0 client.16209 ms_handle_reset on v2:172.21.15.155:6838/2045382457 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 --2- 172.21.15.62:0/493647201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b30e5846a0 0x55b30e595550 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 >> 172.21.15.62:0/493647201 conn(0x55b30e580c30 msgr2=0x55b30e59a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 shutdown_connections 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 1 -- 172.21.15.62:0/493647201 wait complete. 2024-08-16T19:26:12.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.156+0000 7f7169a93740 20 client.16209 trim_cache size 0 max 16384 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 75, 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout: "leader": 24476, 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout: "set_version": 56, 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout: "sets": {} 2024-08-16T19:26:12.167 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:12.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 msgr2=0x7f714c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 0x7f714c0a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f715801c000 tx=0x7f715801e000 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 msgr2=0x7f71601981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f715c009230 tx=0x7f715c004270 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 shutdown_connections 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f714c0a2cf0 0x7f714c0a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71601078b0 0x7f716019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7160102250 0x7f71601986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 --2- 172.21.15.62:0/3108746652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601018b0 0x7f71601981b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:12.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 >> 172.21.15.62:0/3108746652 conn(0x7f71600fd5f0 msgr2=0x7f71600fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 shutdown_connections 2024-08-16T19:26:12.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:12.157+0000 7f7167695640 1 -- 172.21.15.62:0/3108746652 wait complete. 2024-08-16T19:26:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:11 smithi155 ceph-mon[37413]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-08-16T19:26:12.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:11 smithi155 ceph-mon[37413]: osdmap e113: 12 total, 12 up, 12 in 2024-08-16T19:26:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:11 smithi161 ceph-mon[37576]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.7 MiB/s wr, 573 op/s 2024-08-16T19:26:12.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:11 smithi161 ceph-mon[37576]: osdmap e113: 12 total, 12 up, 12 in 2024-08-16T19:26:12.860 INFO:tasks.quiescer:done joining Quiescers 2024-08-16T19:26:12.860 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-16T19:26:12.873 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-16T19:26:12.873 INFO:teuthology.task.exec:Executing custom commands... 2024-08-16T19:26:12.873 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:26:12.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-16T19:26:12.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-16T19:26:12.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-16T19:26:13.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 -- 172.21.15.62:0/2904535641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 msgr2=0x7fa640106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 --2- 172.21.15.62:0/2904535641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa640106ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa638009930 tx=0x7fa63802a420 comp rx=0 tx=0).stop 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 -- 172.21.15.62:0/2904535641 shutdown_connections 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 --2- 172.21.15.62:0/2904535641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa640101080 0x7fa64010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 --2- 172.21.15.62:0/2904535641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6401006e0 0x7fa640100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 --2- 172.21.15.62:0/2904535641 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa640106ad0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 -- 172.21.15.62:0/2904535641 >> 172.21.15.62:0/2904535641 conn(0x7fa6400776c0 msgr2=0x7fa6400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 -- 172.21.15.62:0/2904535641 shutdown_connections 2024-08-16T19:26:13.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.116+0000 7fa644d44640 1 -- 172.21.15.62:0/2904535641 wait complete. 2024-08-16T19:26:13.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 Processor -- start 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 -- start start 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6401006e0 0x7fa640198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 0x7fa640199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa640110b30 con 0x7fa6401066f0 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa6401109b0 con 0x7fa640101080 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa644d44640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa640110cb0 con 0x7fa6401006e0 2024-08-16T19:26:13.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa63ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa63ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50620/0 (socket says 172.21.15.62:50620) 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa63ed9d640 1 -- 172.21.15.62:0/3244166038 learned_addr learned my addr 172.21.15.62:0/3244166038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.117+0000 7fa63e59c640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6401006e0 0x7fa640198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63dd9b640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 0x7fa640199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 -- 172.21.15.62:0/3244166038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6401006e0 msgr2=0x7fa640198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6401006e0 0x7fa640198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 -- 172.21.15.62:0/3244166038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 msgr2=0x7fa640199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 0x7fa640199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 -- 172.21.15.62:0/3244166038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6401a4480 con 0x7fa6401066f0 2024-08-16T19:26:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63dd9b640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 0x7fa640199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa63ed9d640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa63000eb10 tx=0x7fa63000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630004b20 con 0x7fa6401066f0 2024-08-16T19:26:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6401a4650 con 0x7fa6401066f0 2024-08-16T19:26:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa630004cc0 con 0x7fa6401066f0 2024-08-16T19:26:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630010760 con 0x7fa6401066f0 2024-08-16T19:26:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.118+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6401a4b70 con 0x7fa6401066f0 2024-08-16T19:26:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.119+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa640100b40 con 0x7fa6401066f0 2024-08-16T19:26:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.121+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa630005020 con 0x7fa6401066f0 2024-08-16T19:26:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.121+0000 7fa6377fe640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 0x7fa62c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.121+0000 7fa63e59c640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 0x7fa62c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.121+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fa6300cff20 con 0x7fa6401066f0 2024-08-16T19:26:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.122+0000 7fa63e59c640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 0x7fa62c0a5260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa638009900 tx=0x7fa6380058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.125+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa63009c750 con 0x7fa6401066f0 2024-08-16T19:26:13.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.337+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa640103bf0 con 0x7fa62c0a2da0 2024-08-16T19:26:13.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.346+0000 7fa6377fe640 1 -- 172.21.15.62:0/3244166038 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fa640103bf0 con 0x7fa62c0a2da0 2024-08-16T19:26:13.347 INFO:teuthology.orchestra.run.smithi062.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-16T00:00:00", "created": "2024-08-16T19:10:29", "first": "2024-08-16T19:11:00", "last": "2024-08-16T19:26:00", "last_pruned": "2024-08-16T19:26:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 msgr2=0x7fa62c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 0x7fa62c0a5260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa638009900 tx=0x7fa6380058d0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 msgr2=0x7fa6401a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa63000eb10 tx=0x7fa63000c2d0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 shutdown_connections 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa62c0a2da0 0x7fa62c0a5260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6401066f0 0x7fa6401a3f40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa640101080 0x7fa640199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 --2- 172.21.15.62:0/3244166038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6401006e0 0x7fa640198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 >> 172.21.15.62:0/3244166038 conn(0x7fa6400776c0 msgr2=0x7fa6401034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:13.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 shutdown_connections 2024-08-16T19:26:13.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.348+0000 7fa644d44640 1 -- 172.21.15.62:0/3244166038 wait complete. 2024-08-16T19:26:13.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-16T19:26:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 -- 172.21.15.62:0/1524812778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98100700 msgr2=0x7fae98100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 --2- 172.21.15.62:0/1524812778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98100700 0x7fae98100b60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009930 tx=0x7fae8c02a420 comp rx=0 tx=0).stop 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 -- 172.21.15.62:0/1524812778 shutdown_connections 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 --2- 172.21.15.62:0/1524812778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae9810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 --2- 172.21.15.62:0/1524812778 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98100700 0x7fae98100b60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 --2- 172.21.15.62:0/1524812778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98106710 0x7fae98106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 -- 172.21.15.62:0/1524812778 >> 172.21.15.62:0/1524812778 conn(0x7fae980fc420 msgr2=0x7fae980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 -- 172.21.15.62:0/1524812778 shutdown_connections 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.466+0000 7fae9d08f640 1 -- 172.21.15.62:0/1524812778 wait complete. 2024-08-16T19:26:13.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 Processor -- start 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 -- start start 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98100700 0x7fae981980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae98198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111c40 con 0x7fae98106710 2024-08-16T19:26:13.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111ac0 con 0x7fae981010a0 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9d08f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111dc0 con 0x7fae98100700 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9759e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50626/0 (socket says 172.21.15.62:50626) 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae9759e640 1 -- 172.21.15.62:0/3666900425 learned_addr learned my addr 172.21.15.62:0/3666900425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.467+0000 7fae96d9d640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98100700 0x7fae981980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9659c640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae98198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 -- 172.21.15.62:0/3666900425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98100700 msgr2=0x7fae981980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98100700 0x7fae981980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 -- 172.21.15.62:0/3666900425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 msgr2=0x7fae98198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae98198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 -- 172.21.15.62:0/3666900425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819d0c0 con 0x7fae98106710 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9659c640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae98198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9759e640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fae9000cad0 tx=0x7fae9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000ae20 con 0x7fae98106710 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9819d3b0 con 0x7fae98106710 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae90005020 con 0x7fae98106710 2024-08-16T19:26:13.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000f6f0 con 0x7fae98106710 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.468+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a4c40 con 0x7fae98106710 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.469+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae98101e40 con 0x7fae98106710 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.470+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae900051d0 con 0x7fae98106710 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.471+0000 7fae7ffff640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 0x7fae800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.471+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fae900cf900 con 0x7fae98106710 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.471+0000 7fae96d9d640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 0x7fae800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.471+0000 7fae96d9d640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 0x7fae800a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b50 tx=0x7fae88005f90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.475+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fae90014030 con 0x7fae98106710 2024-08-16T19:26:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.685+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 --> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fae980fd2d0 con 0x7fae800a2da0 2024-08-16T19:26:13.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.694+0000 7fae7ffff640 1 -- 172.21.15.62:0/3666900425 <== mgr.15297 v2:172.21.15.62:6800/2270350447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fae980fd2d0 con 0x7fae800a2da0 2024-08-16T19:26:13.695 INFO:teuthology.orchestra.run.smithi062.stdout:/ 1m 6m3h 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.695+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 msgr2=0x7fae800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.695+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 0x7fae800a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae88009b50 tx=0x7fae88005f90 comp rx=0 tx=0).stop 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.695+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 msgr2=0x7fae9819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.695+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fae9000cad0 tx=0x7fae9000cf90 comp rx=0 tx=0).stop 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 shutdown_connections 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fae800a2da0 0x7fae800a5260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae98106710 0x7fae9819c990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae981010a0 0x7fae98198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 --2- 172.21.15.62:0/3666900425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae98100700 0x7fae981980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 >> 172.21.15.62:0/3666900425 conn(0x7fae980fc420 msgr2=0x7fae980fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 shutdown_connections 2024-08-16T19:26:13.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.696+0000 7fae9d08f640 1 -- 172.21.15.62:0/3666900425 wait complete. 2024-08-16T19:26:13.707 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-16T19:26:13.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-16T19:26:13.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 -- 172.21.15.62:0/3346444509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe58101050 msgr2=0x7efe5810d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/3346444509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe58101050 0x7efe5810d580 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efe50009930 tx=0x7efe5002a420 comp rx=0 tx=0).stop 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 -- 172.21.15.62:0/3346444509 shutdown_connections 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/3346444509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe58101050 0x7efe5810d580 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/3346444509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 0x7efe58100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/3346444509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe581066c0 0x7efe58106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 -- 172.21.15.62:0/3346444509 >> 172.21.15.62:0/3346444509 conn(0x7efe58077740 msgr2=0x7efe580feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 -- 172.21.15.62:0/3346444509 shutdown_connections 2024-08-16T19:26:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.824+0000 7efe5e0ae640 1 -- 172.21.15.62:0/3346444509 wait complete. 2024-08-16T19:26:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 Processor -- start 2024-08-16T19:26:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 -- start start 2024-08-16T19:26:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 0x7efe581980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe58101050 0x7efe581985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe58110b00 con 0x7efe581066c0 2024-08-16T19:26:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7efe58110980 con 0x7efe58101050 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe5e0ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe58110c80 con 0x7efe581006b0 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.825+0000 7efe57fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50654/0 (socket says 172.21.15.62:50654) 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 -- 172.21.15.62:0/1926535158 learned_addr learned my addr 172.21.15.62:0/1926535158 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe56ffd640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe58101050 0x7efe581985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe577fe640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 0x7efe581980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 -- 172.21.15.62:0/1926535158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 msgr2=0x7efe581980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 0x7efe581980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 -- 172.21.15.62:0/1926535158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe58101050 msgr2=0x7efe581985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe58101050 0x7efe581985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 -- 172.21.15.62:0/1926535158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5819d0a0 con 0x7efe581066c0 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe57fff640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efe50005ab0 tx=0x7efe500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5003f030 con 0x7efe581066c0 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500044c0 con 0x7efe581066c0 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5819d330 con 0x7efe581066c0 2024-08-16T19:26:13.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50035070 con 0x7efe581066c0 2024-08-16T19:26:13.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.826+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe581a4c20 con 0x7efe581066c0 2024-08-16T19:26:13.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.827+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe58100b10 con 0x7efe581066c0 2024-08-16T19:26:13.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.828+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efe50042030 con 0x7efe581066c0 2024-08-16T19:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.829+0000 7efe54ff9640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 0x7efe400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.829+0000 7efe577fe640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 0x7efe400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.829+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7efe500efbb0 con 0x7efe581066c0 2024-08-16T19:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.830+0000 7efe577fe640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 0x7efe400a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe48005fd0 tx=0x7efe48009500 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:13.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:13.833+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efe500bbd90 con 0x7efe581066c0 2024-08-16T19:26:14.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.023+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe581a4df0 con 0x7efe581066c0 2024-08-16T19:26:14.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.025+0000 7efe54ff9640 1 -- 172.21.15.62:0/1926535158 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+25389 (secure 0 0 0) 0x7efe500c0090 con 0x7efe581066c0 2024-08-16T19:26:14.026 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:14.026 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":113,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:26:11.635281+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/151448061":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/795341867":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1715350407":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/439686452":"2024-08-17T19:10:28.302138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 msgr2=0x7efe400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 0x7efe400a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe48005fd0 tx=0x7efe48009500 comp rx=0 tx=0).stop 2024-08-16T19:26:14.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 msgr2=0x7efe5819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efe50005ab0 tx=0x7efe500040c0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 shutdown_connections 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7efe400a2d00 0x7efe400a51c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe581066c0 0x7efe5819c970 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe58101050 0x7efe581985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 --2- 172.21.15.62:0/1926535158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe581006b0 0x7efe581980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 >> 172.21.15.62:0/1926535158 conn(0x7efe58077740 msgr2=0x7efe58069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.027+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 shutdown_connections 2024-08-16T19:26:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.028+0000 7efe5e0ae640 1 -- 172.21.15.62:0/1926535158 wait complete. 2024-08-16T19:26:14.040 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-16T19:03:58.075413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-16T19:08:01.384527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-16T19:08:03.560289+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-16T19:08:10.039515+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-16T19:26:14.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-16T19:26:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 -- 172.21.15.62:0/3834447911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601032c0 msgr2=0x7fc86010b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 --2- 172.21.15.62:0/3834447911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601032c0 0x7fc86010b7a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009900 tx=0x7fc84c02a480 comp rx=0 tx=0).stop 2024-08-16T19:26:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 -- 172.21.15.62:0/3834447911 shutdown_connections 2024-08-16T19:26:14.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 --2- 172.21.15.62:0/3834447911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc86010bdc0 0x7fc86010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 --2- 172.21.15.62:0/3834447911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601032c0 0x7fc86010b7a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 --2- 172.21.15.62:0/3834447911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8601029a0 0x7fc860102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 -- 172.21.15.62:0/3834447911 >> 172.21.15.62:0/3834447911 conn(0x7fc8600fc3d0 msgr2=0x7fc8600fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 -- 172.21.15.62:0/3834447911 shutdown_connections 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.148+0000 7fc866b55640 1 -- 172.21.15.62:0/3834447911 wait complete. 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 Processor -- start 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 -- start start 2024-08-16T19:26:14.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8601032c0 0x7fc860198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86010bdc0 0x7fc86019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc860111c30 con 0x7fc8601029a0 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fc860111ab0 con 0x7fc86010bdc0 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc866b55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc860111db0 con 0x7fc8601032c0 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc8648f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc8648f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50660/0 (socket says 172.21.15.62:50660) 2024-08-16T19:26:14.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.149+0000 7fc8648f1640 1 -- 172.21.15.62:0/3645557854 learned_addr learned my addr 172.21.15.62:0/3645557854 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8650f2640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86010bdc0 0x7fc86019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc85ffff640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8601032c0 0x7fc860198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 -- 172.21.15.62:0/3645557854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8601032c0 msgr2=0x7fc860198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8601032c0 0x7fc860198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 -- 172.21.15.62:0/3645557854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86010bdc0 msgr2=0x7fc86019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86010bdc0 0x7fc86019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 -- 172.21.15.62:0/3645557854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86019d130 con 0x7fc8601029a0 2024-08-16T19:26:14.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc8648f1640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d950 tx=0x7fc85400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854017030 con 0x7fc8601029a0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86019d420 con 0x7fc8601029a0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc854004020 con 0x7fc8601029a0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8540047a0 con 0x7fc8601029a0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.150+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8601a4cb0 con 0x7fc8601029a0 2024-08-16T19:26:14.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.151+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc860102d80 con 0x7fc8601029a0 2024-08-16T19:26:14.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.152+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8540041d0 con 0x7fc8601029a0 2024-08-16T19:26:14.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.153+0000 7fc85dffb640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 0x7fc8500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.153+0000 7fc85ffff640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 0x7fc8500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.153+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fc8540cf300 con 0x7fc8601029a0 2024-08-16T19:26:14.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.154+0000 7fc85ffff640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 0x7fc8500a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c005a20 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.157+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc85409bb30 con 0x7fc8601029a0 2024-08-16T19:26:14.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:13 smithi062 ceph-mon[32993]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-08-16T19:26:14.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:13 smithi155 ceph-mon[37413]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-08-16T19:26:14.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.342+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc86002d5e0 con 0x7fc8601029a0 2024-08-16T19:26:14.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.342+0000 7fc85dffb640 1 -- 172.21.15.62:0/3645557854 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fc85409b430 con 0x7fc8601029a0 2024-08-16T19:26:14.344 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 msgr2=0x7fc8500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 0x7fc8500a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c005a20 comp rx=0 tx=0).stop 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 msgr2=0x7fc860198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d950 tx=0x7fc85400de10 comp rx=0 tx=0).stop 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 shutdown_connections 2024-08-16T19:26:14.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fc8500a2d00 0x7fc8500a51c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86010bdc0 0x7fc86019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8601032c0 0x7fc860198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 --2- 172.21.15.62:0/3645557854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8601029a0 0x7fc860198140 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.344+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 >> 172.21.15.62:0/3645557854 conn(0x7fc8600fc3d0 msgr2=0x7fc8600fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.345+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 shutdown_connections 2024-08-16T19:26:14.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.345+0000 7fc866b55640 1 -- 172.21.15.62:0/3645557854 wait complete. 2024-08-16T19:26:14.356 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-16T19:26:14.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:13 smithi161 ceph-mon[37576]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-08-16T19:26:14.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- 172.21.15.62:0/646129507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 msgr2=0x7fbbac106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 --2- 172.21.15.62:0/646129507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac106ac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009930 tx=0x7fbb9c02a420 comp rx=0 tx=0).stop 2024-08-16T19:26:14.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- 172.21.15.62:0/646129507 shutdown_connections 2024-08-16T19:26:14.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 --2- 172.21.15.62:0/646129507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac101070 0x7fbbac10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 --2- 172.21.15.62:0/646129507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac1006d0 0x7fbbac100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 --2- 172.21.15.62:0/646129507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac106ac0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- 172.21.15.62:0/646129507 >> 172.21.15.62:0/646129507 conn(0x7fbbac0fc3d0 msgr2=0x7fbbac0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- 172.21.15.62:0/646129507 shutdown_connections 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- 172.21.15.62:0/646129507 wait complete. 2024-08-16T19:26:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 Processor -- start 2024-08-16T19:26:14.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.466+0000 7fbbb3274640 1 -- start start 2024-08-16T19:26:14.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac1006d0 0x7fbbac198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac101070 0x7fbbac198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111c10 con 0x7fbbac1066e0 2024-08-16T19:26:14.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111a90 con 0x7fbbac101070 2024-08-16T19:26:14.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac111d90 con 0x7fbbac1006d0 2024-08-16T19:26:14.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 -- 172.21.15.62:0/2702121625 learned_addr learned my addr 172.21.15.62:0/2702121625 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1010640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac1006d0 0x7fbbac198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb080f640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac101070 0x7fbbac198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 -- 172.21.15.62:0/2702121625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac1006d0 msgr2=0x7fbbac198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac1006d0 0x7fbbac198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 -- 172.21.15.62:0/2702121625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac101070 msgr2=0x7fbbac198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac101070 0x7fbbac198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 -- 172.21.15.62:0/2702121625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19d110 con 0x7fbbac1066e0 2024-08-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb1811640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009fd0 tx=0x7fbba400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4017070 con 0x7fbbac1066e0 2024-08-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba4003e50 con 0x7fbbac1066e0 2024-08-16T19:26:14.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac19d400 con 0x7fbbac1066e0 2024-08-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.467+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba40106c0 con 0x7fbbac1066e0 2024-08-16T19:26:14.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.468+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a4c90 con 0x7fbbac1066e0 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.469+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbac100b30 con 0x7fbbac1066e0 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.470+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbba4003970 con 0x7fbbac1066e0 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.470+0000 7fbbaa7fc640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 0x7fbb940a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.471+0000 7fbbb1010640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 0x7fbb940a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.471+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fbba40cf420 con 0x7fbbac1066e0 2024-08-16T19:26:14.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.471+0000 7fbbb1010640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 0x7fbb940a5240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c0058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.474+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbba409b680 con 0x7fbbac1066e0 2024-08-16T19:26:14.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.660+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbac02d5e0 con 0x7fbbac1066e0 2024-08-16T19:26:14.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.661+0000 7fbbaa7fc640 1 -- 172.21.15.62:0/2702121625 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fbba409f980 con 0x7fbbac1066e0 2024-08-16T19:26:14.662 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-08-16T19:26:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 msgr2=0x7fbb940a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 0x7fbb940a5240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c0058d0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 msgr2=0x7fbbac19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009fd0 tx=0x7fbba400b260 comp rx=0 tx=0).stop 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 shutdown_connections 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fbb940a2d80 0x7fbb940a5240 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbac1066e0 0x7fbbac19c9e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbbac101070 0x7fbbac198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 --2- 172.21.15.62:0/2702121625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbac1006d0 0x7fbbac198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 >> 172.21.15.62:0/2702121625 conn(0x7fbbac0fc3d0 msgr2=0x7fbbac0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 shutdown_connections 2024-08-16T19:26:14.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.663+0000 7fbbb3274640 1 -- 172.21.15.62:0/2702121625 wait complete. 2024-08-16T19:26:14.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-16T19:26:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 -- 172.21.15.62:0/4009458514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9681006b0 msgr2=0x7ff968100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 --2- 172.21.15.62:0/4009458514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9681006b0 0x7ff968100b10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009930 tx=0x7ff95c02a420 comp rx=0 tx=0).stop 2024-08-16T19:26:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 -- 172.21.15.62:0/4009458514 shutdown_connections 2024-08-16T19:26:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 --2- 172.21.15.62:0/4009458514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff968101050 0x7ff96810d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 --2- 172.21.15.62:0/4009458514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff9681006b0 0x7ff968100b10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 --2- 172.21.15.62:0/4009458514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff968106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 -- 172.21.15.62:0/4009458514 >> 172.21.15.62:0/4009458514 conn(0x7ff968077700 msgr2=0x7ff9680feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 -- 172.21.15.62:0/4009458514 shutdown_connections 2024-08-16T19:26:14.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.783+0000 7ff96cc30640 1 -- 172.21.15.62:0/4009458514 wait complete. 2024-08-16T19:26:14.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 Processor -- start 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 -- start start 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9681006b0 0x7ff968198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff96819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff968110b00 con 0x7ff968101050 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7ff968110980 con 0x7ff9681066c0 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96cc30640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff968110c80 con 0x7ff9681006b0 2024-08-16T19:26:14.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50694/0 (socket says 172.21.15.62:50694) 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 -- 172.21.15.62:0/910918546 learned_addr learned my addr 172.21.15.62:0/910918546 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff96659c640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9681006b0 0x7ff968198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 -- 172.21.15.62:0/910918546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9681006b0 msgr2=0x7ff968198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9681006b0 0x7ff968198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 -- 172.21.15.62:0/910918546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 msgr2=0x7ff96819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff966d9d640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff96819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff96819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff965d9b640 1 -- 172.21.15.62:0/910918546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96819d140 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.784+0000 7ff966d9d640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff96819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff965d9b640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009230 tx=0x7ff95c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c03f030 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff95c003ad0 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96819d3d0 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c035070 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.785+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681a4cc0 con 0x7ff968101050 2024-08-16T19:26:14.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.786+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff968100b10 con 0x7ff968101050 2024-08-16T19:26:14.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.787+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff95c042030 con 0x7ff968101050 2024-08-16T19:26:14.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.788+0000 7ff9577fe640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 0x7ff9500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:14.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.788+0000 7ff96659c640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 0x7ff9500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:14.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.788+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7ff95c0f00f0 con 0x7ff968101050 2024-08-16T19:26:14.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.788+0000 7ff96659c640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 0x7ff9500a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff95800bb60 tx=0x7ff95800b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:14.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.791+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff95c0bc920 con 0x7ff968101050 2024-08-16T19:26:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.979+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9681a4ec0 con 0x7ff968101050 2024-08-16T19:26:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.982+0000 7ff9577fe640 1 -- 172.21.15.62:0/910918546 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7ff95c0bc220 con 0x7ff968101050 2024-08-16T19:26:14.982 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-08-16T19:26:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 msgr2=0x7ff9500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 0x7ff9500a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff95800bb60 tx=0x7ff95800b390 comp rx=0 tx=0).stop 2024-08-16T19:26:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 msgr2=0x7ff968198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009230 tx=0x7ff95c004270 comp rx=0 tx=0).stop 2024-08-16T19:26:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 shutdown_connections 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7ff9500a2d00 0x7ff9500a51c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9681066c0 0x7ff96819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff968101050 0x7ff968198690 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 --2- 172.21.15.62:0/910918546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9681006b0 0x7ff968198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 >> 172.21.15.62:0/910918546 conn(0x7ff968077700 msgr2=0x7ff9680694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:14.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 shutdown_connections 2024-08-16T19:26:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:14.984+0000 7ff96cc30640 1 -- 172.21.15.62:0/910918546 wait complete. 2024-08-16T19:26:14.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-16T19:26:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 -- 172.21.15.62:0/744102732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c1032c0 msgr2=0x7fb70c10b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 --2- 172.21.15.62:0/744102732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c1032c0 0x7fb70c10b7a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb700009900 tx=0x7fb70002a480 comp rx=0 tx=0).stop 2024-08-16T19:26:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 -- 172.21.15.62:0/744102732 shutdown_connections 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 --2- 172.21.15.62:0/744102732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c10bdc0 0x7fb70c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 --2- 172.21.15.62:0/744102732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c1032c0 0x7fb70c10b7a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 --2- 172.21.15.62:0/744102732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1029a0 0x7fb70c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 -- 172.21.15.62:0/744102732 >> 172.21.15.62:0/744102732 conn(0x7fb70c0fc3d0 msgr2=0x7fb70c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 -- 172.21.15.62:0/744102732 shutdown_connections 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 -- 172.21.15.62:0/744102732 wait complete. 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.104+0000 7fb711364640 1 Processor -- start 2024-08-16T19:26:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 -- start start 2024-08-16T19:26:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c1029a0 0x7fb70c1005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1032c0 0x7fb70c100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c111c20 con 0x7fb70c10bdc0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c111aa0 con 0x7fb70c1032c0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb711364640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c111da0 con 0x7fb70c1029a0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50708/0 (socket says 172.21.15.62:50708) 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 -- 172.21.15.62:0/6466589 learned_addr learned my addr 172.21.15.62:0/6466589 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70a7fc640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1032c0 0x7fb70c100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70affd640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c1029a0 0x7fb70c1005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 -- 172.21.15.62:0/6466589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c1029a0 msgr2=0x7fb70c1005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c1029a0 0x7fb70c1005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 -- 172.21.15.62:0/6466589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1032c0 msgr2=0x7fb70c100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1032c0 0x7fb70c100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 -- 172.21.15.62:0/6466589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c1a6680 con 0x7fb70c10bdc0 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.105+0000 7fb70b7fe640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb70400cad0 tx=0x7fb70400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.106+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400ad60 con 0x7fb70c10bdc0 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.106+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb704005020 con 0x7fb70c10bdc0 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.106+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c1a6970 con 0x7fb70c10bdc0 2024-08-16T19:26:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.106+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400f6b0 con 0x7fb70c10bdc0 2024-08-16T19:26:15.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.106+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a6e40 con 0x7fb70c10bdc0 2024-08-16T19:26:15.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.107+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c102d80 con 0x7fb70c10bdc0 2024-08-16T19:26:15.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.108+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb704005560 con 0x7fb70c10bdc0 2024-08-16T19:26:15.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.109+0000 7fb6f3fff640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 0x7fb6f40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.109+0000 7fb70affd640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 0x7fb6f40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.109+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fb7040cf830 con 0x7fb70c10bdc0 2024-08-16T19:26:15.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.109+0000 7fb70affd640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 0x7fb6f40a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009990 tx=0x7fb6fc009110 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:15.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.113+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb704014030 con 0x7fb70c10bdc0 2024-08-16T19:26:15.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:14 smithi062 ceph-mon[32993]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:14 smithi062 ceph-mon[32993]: from='client.16221 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:14 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1926535158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:14 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/3645557854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:14 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2702121625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.299+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb70c106dc0 con 0x7fb70c10bdc0 2024-08-16T19:26:15.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.300+0000 7fb6f3fff640 1 -- 172.21.15.62:0/6466589 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) ==== 103+0+11 (secure 0 0 0) 0x7fb70409c060 con 0x7fb70c10bdc0 2024-08-16T19:26:15.301 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 msgr2=0x7fb6f40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 0x7fb6f40a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009990 tx=0x7fb6fc009110 comp rx=0 tx=0).stop 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 msgr2=0x7fb70c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb70400cad0 tx=0x7fb70400ced0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 shutdown_connections 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb6f40a2d00 0x7fb6f40a51c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb70c10bdc0 0x7fb70c101070 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70c1032c0 0x7fb70c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 --2- 172.21.15.62:0/6466589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb70c1029a0 0x7fb70c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 >> 172.21.15.62:0/6466589 conn(0x7fb70c0fc3d0 msgr2=0x7fb70c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 shutdown_connections 2024-08-16T19:26:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.302+0000 7fb711364640 1 -- 172.21.15.62:0/6466589 wait complete. 2024-08-16T19:26:15.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-16T19:26:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:14 smithi155 ceph-mon[37413]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:14 smithi155 ceph-mon[37413]: from='client.16221 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:14 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1926535158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:14 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/3645557854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:14 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2702121625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:14 smithi161 ceph-mon[37576]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:14 smithi161 ceph-mon[37576]: from='client.16221 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-16T19:26:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:14 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1926535158' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:14 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/3645557854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:14 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2702121625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-16T19:26:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 -- 172.21.15.62:0/150806555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde70101090 msgr2=0x7fde7010d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 --2- 172.21.15.62:0/150806555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde70101090 0x7fde7010d5c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fde68009cd0 tx=0x7fde6802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 -- 172.21.15.62:0/150806555 shutdown_connections 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 --2- 172.21.15.62:0/150806555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde70101090 0x7fde7010d5c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 --2- 172.21.15.62:0/150806555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde701006f0 0x7fde70100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 --2- 172.21.15.62:0/150806555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 0x7fde70106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 -- 172.21.15.62:0/150806555 >> 172.21.15.62:0/150806555 conn(0x7fde700fc3d0 msgr2=0x7fde700fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 -- 172.21.15.62:0/150806555 shutdown_connections 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.426+0000 7fde76ed3640 1 -- 172.21.15.62:0/150806555 wait complete. 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 Processor -- start 2024-08-16T19:26:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 -- start start 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde70101090 0x7fde70197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 0x7fde7019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fde70110b40 con 0x7fde701006f0 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fde701109c0 con 0x7fde70101090 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde76ed3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fde70110cc0 con 0x7fde70106700 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde74c6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde74c6f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50724/0 (socket says 172.21.15.62:50724) 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.427+0000 7fde74c6f640 1 -- 172.21.15.62:0/1673856355 learned_addr learned my addr 172.21.15.62:0/1673856355 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde75470640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 0x7fde7019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde6ffff640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde70101090 0x7fde70197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 -- 172.21.15.62:0/1673856355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 msgr2=0x7fde7019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 0x7fde7019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 -- 172.21.15.62:0/1673856355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde70101090 msgr2=0x7fde70197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde70101090 0x7fde70197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:15.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 -- 172.21.15.62:0/1673856355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7019c000 con 0x7fde701006f0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde74c6f640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fde60007b50 tx=0x7fde6000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60007c40 con 0x7fde701006f0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7019c2f0 con 0x7fde701006f0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60007de0 con 0x7fde701006f0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6000ae60 con 0x7fde701006f0 2024-08-16T19:26:15.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.428+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7006a0f0 con 0x7fde701006f0 2024-08-16T19:26:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.429+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde70100b50 con 0x7fde701006f0 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.431+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde6000a440 con 0x7fde701006f0 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.431+0000 7fde6dffb640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 0x7fde580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.431+0000 7fde6ffff640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 0x7fde580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.431+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fde600d1070 con 0x7fde701006f0 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.432+0000 7fde6ffff640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 0x7fde580a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde700fe8f0 tx=0x7fde64008040 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.434+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fde60023060 con 0x7fde701006f0 2024-08-16T19:26:15.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.674+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fde70198360 con 0x7fde701006f0 2024-08-16T19:26:15.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.682+0000 7fde6dffb640 1 -- 172.21.15.62:0/1673856355 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740900 (secure 0 0 0) 0x7fde70198360 con 0x7fde701006f0 2024-08-16T19:26:15.693 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:15.694 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:15.694 INFO:teuthology.orchestra.run.smithi062.stdout: "epoch": 25, 2024-08-16T19:26:15.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.694 INFO:teuthology.orchestra.run.smithi062.stdout: "active_gid": 15297, 2024-08-16T19:26:15.695 INFO:teuthology.orchestra.run.smithi062.stdout: "active_name": "x", 2024-08-16T19:26:15.695 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addrs": { 2024-08-16T19:26:15.695 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:15.695 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:15.696 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6800", 2024-08-16T19:26:15.696 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2270350447 2024-08-16T19:26:15.696 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.696 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.697 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v1", 2024-08-16T19:26:15.697 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:6801", 2024-08-16T19:26:15.697 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2270350447 2024-08-16T19:26:15.697 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.698 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:15.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.698 INFO:teuthology.orchestra.run.smithi062.stdout: "active_addr": "172.21.15.62:6801/2270350447", 2024-08-16T19:26:15.698 INFO:teuthology.orchestra.run.smithi062.stdout: "active_change": "2024-08-16T19:10:28.302364+0000", 2024-08-16T19:26:15.698 INFO:teuthology.orchestra.run.smithi062.stdout: "active_mgr_features": 4540701547738038271, 2024-08-16T19:26:15.699 INFO:teuthology.orchestra.run.smithi062.stdout: "available": true, 2024-08-16T19:26:15.699 INFO:teuthology.orchestra.run.smithi062.stdout: "standbys": [ 2024-08-16T19:26:15.699 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.699 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24959, 2024-08-16T19:26:15.699 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "y", 2024-08-16T19:26:15.700 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-08-16T19:26:15.700 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-08-16T19:26:15.700 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-08-16T19:26:15.700 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:15.701 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:15.701 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:15.701 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:15.701 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:15.701 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.702 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.704 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.706 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.707 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:15.708 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.709 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.712 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-08-16T19:26:15.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-08-16T19:26:15.723 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.724 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-08-16T19:26:15.724 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.726 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.728 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.729 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:15.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-08-16T19:26:15.734 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.735 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.736 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-16T19:26:15.737 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-08-16T19:26:15.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:15.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.742 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-08-16T19:26:15.742 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-08-16T19:26:15.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:15.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:15.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.745 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.748 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.749 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.750 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.751 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:15.752 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:15.752 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.752 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-08-16T19:26:15.754 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-08-16T19:26:15.754 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:15.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.756 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:15.756 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:15.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-08-16T19:26:15.757 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-08-16T19:26:15.758 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-08-16T19:26:15.758 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-08-16T19:26:15.758 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-08-16T19:26:15.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.759 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-16T19:26:15.760 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.761 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:15.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:15.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:15.763 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.763 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-16T19:26:15.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.764 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.764 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.764 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.764 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-08-16T19:26:15.764 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-08-16T19:26:15.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.768 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:15.769 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.770 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-08-16T19:26:15.772 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.774 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-08-16T19:26:15.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-08-16T19:26:15.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-16T19:26:15.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.778 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-16T19:26:15.779 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-08-16T19:26:15.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-16T19:26:15.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.782 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-08-16T19:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-08-16T19:26:15.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.785 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.785 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-08-16T19:26:15.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-08-16T19:26:15.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.790 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.792 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-08-16T19:26:15.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-08-16T19:26:15.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.794 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-16T19:26:15.794 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.794 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.795 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-08-16T19:26:15.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.797 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-16T19:26:15.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.799 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.800 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-08-16T19:26:15.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-08-16T19:26:15.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-16T19:26:15.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-08-16T19:26:15.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-08-16T19:26:15.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-08-16T19:26:15.806 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.806 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.806 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.806 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.807 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.808 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.809 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-08-16T19:26:15.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.810 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.810 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.811 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.812 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.813 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:15.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-08-16T19:26:15.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-16T19:26:15.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.817 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-08-16T19:26:15.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-16T19:26:15.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.821 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-08-16T19:26:15.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-08-16T19:26:15.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.822 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.823 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.824 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.825 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.826 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.827 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.828 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:15.829 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.829 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-08-16T19:26:15.830 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.831 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-16T19:26:15.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:15.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.840 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-16T19:26:15.841 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.842 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.843 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.844 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.845 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.846 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:15.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.850 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.854 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.855 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:15.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:15.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.856 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.857 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.857 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.857 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.857 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.857 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.858 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.860 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.860 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.861 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-16T19:26:15.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-16T19:26:15.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.862 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.862 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-16T19:26:15.864 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-16T19:26:15.865 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-16T19:26:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.869 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.870 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-08-16T19:26:15.870 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-08-16T19:26:15.870 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.870 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.871 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.872 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-08-16T19:26:15.873 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-08-16T19:26:15.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-16T19:26:15.875 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-16T19:26:15.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.878 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.878 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-08-16T19:26:15.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-08-16T19:26:15.882 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-16T19:26:15.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-08-16T19:26:15.888 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.889 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.891 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.892 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.893 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.894 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-08-16T19:26:15.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-08-16T19:26:15.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.897 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-08-16T19:26:15.898 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-08-16T19:26:15.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.900 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.900 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.900 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-08-16T19:26:15.900 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-08-16T19:26:15.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.901 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.901 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.901 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.902 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-16T19:26:15.902 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.902 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.903 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-08-16T19:26:15.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-08-16T19:26:15.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.904 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.904 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.905 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:15.906 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:15.912 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.913 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-08-16T19:26:15.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.917 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.919 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-16T19:26:15.922 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.923 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.925 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-08-16T19:26:15.926 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.927 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.928 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-16T19:26:15.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.930 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.932 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.932 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.932 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.932 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.932 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.933 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.934 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-16T19:26:15.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.937 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.938 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.940 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.942 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.942 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-16T19:26:15.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-16T19:26:15.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.943 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.944 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.949 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.949 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.949 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.949 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.950 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.955 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-16T19:26:15.957 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.958 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.960 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.961 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.963 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.964 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.965 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-08-16T19:26:15.965 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-16T19:26:15.965 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.966 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-16T19:26:15.967 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.968 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-16T19:26:15.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-16T19:26:15.971 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-16T19:26:15.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.978 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-16T19:26:15.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-16T19:26:15.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-16T19:26:15.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.981 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.982 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.983 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-08-16T19:26:15.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.986 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.988 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.989 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.990 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.991 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.992 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.995 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-08-16T19:26:15.995 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-16T19:26:15.995 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.995 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.995 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:15.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:15.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:15.998 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-16T19:26:15.998 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-16T19:26:15.998 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:15.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:15.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.000 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-08-16T19:26:16.000 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-16T19:26:16.000 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.000 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-16T19:26:16.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-16T19:26:16.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.005 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.005 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.005 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.005 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.009 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-08-16T19:26:16.010 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-08-16T19:26:16.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-08-16T19:26:16.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.016 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-08-16T19:26:16.016 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-08-16T19:26:16.017 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.019 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.020 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.021 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.022 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.026 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.027 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.028 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.029 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-08-16T19:26:16.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.033 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.034 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.036 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-08-16T19:26:16.038 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-08-16T19:26:16.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.042 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-08-16T19:26:16.045 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.046 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.046 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:16.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-08-16T19:26:16.048 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.050 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.053 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.054 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.055 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.058 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.059 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.060 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-08-16T19:26:16.061 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-16T19:26:16.063 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.063 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-16T19:26:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-08-16T19:26:16.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.067 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-08-16T19:26:16.067 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.068 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.069 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.069 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:16.069 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.069 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.069 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.070 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.072 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.074 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-16T19:26:16.077 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.077 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.078 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.078 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.078 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-08-16T19:26:16.078 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.079 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.079 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.079 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.080 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.081 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.082 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.085 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.086 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-08-16T19:26:16.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.091 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.092 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.094 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.095 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.096 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.096 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-08-16T19:26:16.096 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-08-16T19:26:16.096 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.097 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.097 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-08-16T19:26:16.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-16T19:26:16.098 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-08-16T19:26:16.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.101 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-08-16T19:26:16.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.103 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.104 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.105 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.106 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.106 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.109 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.110 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.110 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.115 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.117 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-08-16T19:26:16.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-08-16T19:26:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-08-16T19:26:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.122 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-08-16T19:26:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-08-16T19:26:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.124 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.125 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-08-16T19:26:16.126 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.127 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.128 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.129 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.130 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.133 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.134 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.136 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.137 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.138 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.140 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.141 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.142 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.145 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.146 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.146 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.146 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.147 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-08-16T19:26:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-16T19:26:16.154 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.155 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.156 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.157 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.157 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.157 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.162 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.162 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.162 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.165 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.166 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-08-16T19:26:16.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.168 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.169 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.170 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.172 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.173 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-08-16T19:26:16.178 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-08-16T19:26:16.178 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.178 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.178 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.179 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-08-16T19:26:16.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-08-16T19:26:16.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.182 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-08-16T19:26:16.183 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-08-16T19:26:16.183 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.183 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.185 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-08-16T19:26:16.186 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.190 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.191 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.192 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.193 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.195 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.197 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.197 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.197 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.197 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.198 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.198 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.198 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.198 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.199 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.200 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.201 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.205 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.208 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.209 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.210 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.210 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.210 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.210 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.210 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.213 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.214 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-08-16T19:26:16.214 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.216 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.218 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.220 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.222 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.222 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.222 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.222 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.223 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.224 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.224 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.224 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.225 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.226 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-08-16T19:26:16.227 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-16T19:26:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.229 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.230 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.231 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.232 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.233 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.235 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:16.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-08-16T19:26:16.239 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.240 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.241 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.242 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.243 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.243 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.244 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.245 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.247 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.248 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.249 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.250 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.251 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.253 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.254 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.255 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.256 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.261 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-08-16T19:26:16.265 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.266 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.267 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.268 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.271 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-08-16T19:26:16.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.277 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.278 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-08-16T19:26:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.279 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-08-16T19:26:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-08-16T19:26:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.280 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.281 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.282 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.284 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.285 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.286 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.288 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.291 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.292 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.293 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-08-16T19:26:16.296 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.297 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.298 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.299 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.300 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.301 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-08-16T19:26:16.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.303 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.304 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.306 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.307 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.308 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.310 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.311 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.312 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-08-16T19:26:16.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-08-16T19:26:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-08-16T19:26:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.322 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:16.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-16T19:26:16.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-08-16T19:26:16.325 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.326 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.327 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.328 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.329 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.330 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:16.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.332 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-08-16T19:26:16.333 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.334 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.335 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.337 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.339 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.341 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.342 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-08-16T19:26:16.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.349 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.353 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.356 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.357 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.360 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.363 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:16.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.366 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:16.367 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.368 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.369 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.370 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.371 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.372 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.373 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.374 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.376 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.377 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.378 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.379 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.380 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.381 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.382 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.383 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.386 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.388 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.390 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-08-16T19:26:16.394 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.397 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.398 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.399 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.400 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.401 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.402 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.403 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.408 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-08-16T19:26:16.409 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.411 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.412 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.413 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.418 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.419 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.420 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.421 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.422 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-08-16T19:26:16.423 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.424 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.428 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.429 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-08-16T19:26:16.431 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.432 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.438 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.439 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.440 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.441 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.443 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.444 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.447 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.448 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.449 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.450 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.451 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.453 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.454 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.455 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.457 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-08-16T19:26:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.458 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.460 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.461 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.465 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.466 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.467 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.469 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-08-16T19:26:16.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.472 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.475 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.476 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.480 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.483 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-08-16T19:26:16.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-16T19:26:16.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-08-16T19:26:16.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-16T19:26:16.487 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-16T19:26:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.494 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.495 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-08-16T19:26:16.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-08-16T19:26:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-16T19:26:16.498 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.499 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-08-16T19:26:16.503 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-08-16T19:26:16.504 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.505 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.506 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.507 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.508 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.509 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.510 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.511 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.512 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.513 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.515 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.516 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.517 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.518 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.519 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-08-16T19:26:16.520 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.521 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-16T19:26:16.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.525 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.528 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.529 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.530 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.532 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.534 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.536 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.537 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.538 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.539 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.541 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.544 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.545 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.548 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-08-16T19:26:16.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-08-16T19:26:16.552 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-16T19:26:16.553 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.554 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.555 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.556 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.557 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.558 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:16.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.561 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.562 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.567 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.569 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.570 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-08-16T19:26:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-08-16T19:26:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.571 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.573 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-08-16T19:26:16.575 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.576 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.577 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "gid": 24970, 2024-08-16T19:26:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "z", 2024-08-16T19:26:16.578 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_features": 4540701547738038271, 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.579 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.580 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.581 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.582 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.583 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.584 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.585 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.586 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.588 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-08-16T19:26:16.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-08-16T19:26:16.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.600 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-08-16T19:26:16.601 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.602 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.603 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.604 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-08-16T19:26:16.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-08-16T19:26:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.609 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-08-16T19:26:16.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:16.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.612 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-08-16T19:26:16.613 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-08-16T19:26:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-16T19:26:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.615 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.616 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-16T19:26:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-16T19:26:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.617 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-08-16T19:26:16.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.619 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-08-16T19:26:16.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-08-16T19:26:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-08-16T19:26:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:16.622 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.623 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.625 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.629 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:16.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.635 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.637 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-08-16T19:26:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-08-16T19:26:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:16.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-08-16T19:26:16.644 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.645 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.646 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-08-16T19:26:16.647 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.648 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-08-16T19:26:16.649 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.650 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.652 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-16T19:26:16.653 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-16T19:26:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-08-16T19:26:16.654 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.655 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-08-16T19:26:16.656 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.657 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.658 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.659 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-16T19:26:16.660 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.661 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.662 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-16T19:26:16.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-08-16T19:26:16.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-08-16T19:26:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.671 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.672 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.673 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.674 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.675 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-08-16T19:26:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-16T19:26:16.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-16T19:26:16.681 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-16T19:26:16.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-08-16T19:26:16.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-08-16T19:26:16.687 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-16T19:26:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.688 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.689 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-08-16T19:26:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.691 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-08-16T19:26:16.693 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.694 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.695 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-16T19:26:16.696 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-08-16T19:26:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-08-16T19:26:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.698 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.699 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-08-16T19:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-08-16T19:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.703 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.705 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.707 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-08-16T19:26:16.708 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-08-16T19:26:16.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-08-16T19:26:16.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-08-16T19:26:16.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.717 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.718 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-08-16T19:26:16.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.721 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.723 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.724 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-08-16T19:26:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-08-16T19:26:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.725 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.726 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.727 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.728 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.729 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.730 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.733 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.734 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.735 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-16T19:26:16.736 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.737 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.739 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-08-16T19:26:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.740 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.741 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:16.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-08-16T19:26:16.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-08-16T19:26:16.746 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-08-16T19:26:16.747 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-16T19:26:16.748 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.749 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-16T19:26:16.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-16T19:26:16.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-08-16T19:26:16.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-08-16T19:26:16.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.757 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.758 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.759 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.760 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.761 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.762 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-08-16T19:26:16.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-16T19:26:16.764 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.765 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.767 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-08-16T19:26:16.768 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-08-16T19:26:16.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-08-16T19:26:16.772 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.773 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.774 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.775 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.777 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.780 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.781 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.784 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.788 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-08-16T19:26:16.789 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.791 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-16T19:26:16.792 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.795 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.796 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.797 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.799 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-16T19:26:16.800 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.801 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.805 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.806 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.808 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.809 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.810 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.811 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.812 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.813 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.814 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.815 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.816 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.817 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.818 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.819 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:16.820 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.821 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.822 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.823 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-08-16T19:26:16.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-08-16T19:26:16.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-16T19:26:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-16T19:26:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.834 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.835 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-08-16T19:26:16.836 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.837 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.839 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.840 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-16T19:26:16.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.843 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.844 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.845 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.846 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.847 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.848 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.849 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-16T19:26:16.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.852 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-08-16T19:26:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-16T19:26:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.853 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.854 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-16T19:26:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-16T19:26:16.855 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.857 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-16T19:26:16.858 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-16T19:26:16.858 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.858 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.858 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.858 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.860 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.861 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-16T19:26:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-16T19:26:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.862 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-08-16T19:26:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-08-16T19:26:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-16T19:26:16.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.866 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.867 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-16T19:26:16.868 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.869 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.871 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.872 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.873 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.874 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.875 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.876 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.878 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.879 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.882 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.883 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:16.886 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.890 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.891 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.892 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.893 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.894 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.895 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.896 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-08-16T19:26:16.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-08-16T19:26:16.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.901 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.902 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.904 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.905 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.906 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.907 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.908 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-08-16T19:26:16.909 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-08-16T19:26:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-08-16T19:26:16.910 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.911 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.912 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.913 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.915 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.921 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.923 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.924 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.925 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-08-16T19:26:16.928 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-16T19:26:16.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.932 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.933 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-08-16T19:26:16.935 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.936 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-16T19:26:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.937 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.938 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-08-16T19:26:16.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.941 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.942 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.944 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.948 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.949 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.952 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-08-16T19:26:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-08-16T19:26:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.954 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-08-16T19:26:16.955 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.956 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:16.957 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.958 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-08-16T19:26:16.959 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.960 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.961 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-16T19:26:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.965 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-08-16T19:26:16.966 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.967 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.968 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.969 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.970 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.971 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.972 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.974 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.975 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.978 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-08-16T19:26:16.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:16.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-08-16T19:26:16.983 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-16T19:26:16.984 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-08-16T19:26:16.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-08-16T19:26:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-08-16T19:26:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-08-16T19:26:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.988 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.989 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.990 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:16.991 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:16.993 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:16.994 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:16.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.996 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.997 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:16.998 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:16.999 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.000 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.001 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.002 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.003 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.005 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.010 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.011 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.012 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.013 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.014 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-08-16T19:26:17.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-16T19:26:17.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.019 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.020 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.021 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.023 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.024 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.025 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.026 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.028 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.029 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.031 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.032 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.033 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.035 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.037 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.038 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.039 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.040 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.041 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.042 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.043 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-08-16T19:26:17.044 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.045 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-08-16T19:26:17.046 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-08-16T19:26:17.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-08-16T19:26:17.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-08-16T19:26:17.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.053 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.055 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.056 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.057 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.060 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.061 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.062 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-08-16T19:26:17.065 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.066 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.067 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.068 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.069 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.070 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.073 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.074 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.075 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.076 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.077 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.078 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.079 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.080 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.081 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.082 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.083 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.086 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.087 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.088 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.089 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.090 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.091 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.092 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.093 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.097 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.098 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-08-16T19:26:17.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.102 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.103 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.106 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.107 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.108 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.109 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.110 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.111 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.115 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.116 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.119 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.120 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.121 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.122 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.126 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.127 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.128 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.133 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.134 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.136 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.137 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:17.138 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.139 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-08-16T19:26:17.140 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-16T19:26:17.141 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.142 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.143 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.144 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.145 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.147 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.148 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.149 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.151 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-08-16T19:26:17.152 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-08-16T19:26:17.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.155 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.156 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-08-16T19:26:17.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.161 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.162 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.163 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.164 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.165 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.166 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.167 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.168 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.173 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.174 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.175 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.176 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:17.177 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.178 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.179 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-08-16T19:26:17.180 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.181 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.182 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.186 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.187 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.189 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.190 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.191 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.192 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.193 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.195 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-08-16T19:26:17.197 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.198 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.199 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.200 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.201 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-08-16T19:26:17.202 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.203 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.208 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.210 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.211 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.212 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.213 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.214 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.215 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.218 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.220 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.222 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.224 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.225 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.227 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.228 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.229 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.230 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.232 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.235 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.236 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.240 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.241 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.242 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.243 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.244 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.247 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.248 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.249 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-08-16T19:26:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-08-16T19:26:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-08-16T19:26:17.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.260 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.261 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.262 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.263 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.264 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.265 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.266 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.267 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-08-16T19:26:17.268 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.269 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-08-16T19:26:17.270 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.271 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-08-16T19:26:17.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.273 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.274 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.275 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.276 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.277 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.278 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.279 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.280 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.281 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.282 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.283 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.284 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.285 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.286 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.287 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.288 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.289 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.290 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.291 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.292 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.293 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.294 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.295 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.296 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.297 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.298 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.299 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.300 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.301 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.302 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.303 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.304 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.305 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.306 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.307 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.308 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.309 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.310 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.311 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.312 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.313 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:17.314 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.315 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.316 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.317 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.318 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.319 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.320 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.321 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.322 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.323 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.324 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.325 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.326 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.327 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-08-16T19:26:17.328 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.329 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-08-16T19:26:17.330 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.331 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-16T19:26:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.332 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.333 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-16T19:26:17.334 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.335 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-16T19:26:17.336 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.337 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.338 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.339 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.340 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.342 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.343 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.344 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.345 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.346 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.347 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-08-16T19:26:17.348 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.349 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.350 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.351 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.352 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.353 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.354 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.355 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.356 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.357 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.358 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.359 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-08-16T19:26:17.360 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.361 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-08-16T19:26:17.362 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.363 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.364 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-16T19:26:17.365 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.366 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-08-16T19:26:17.367 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.368 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.369 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.370 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.371 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.372 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.373 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.374 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.375 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.376 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.377 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.378 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.379 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.380 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.381 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.382 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.383 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.384 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.385 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.386 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.387 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-08-16T19:26:17.388 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-16T19:26:17.389 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.390 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.392 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-08-16T19:26:17.393 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.394 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-08-16T19:26:17.396 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.397 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.398 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.399 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:17.400 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.401 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.402 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.403 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.404 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.405 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.406 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.407 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.408 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.409 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.410 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.411 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.412 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.413 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-08-16T19:26:17.414 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.415 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.416 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.417 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.418 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.419 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "modules": [ 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard", 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "iostat", 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "nfs", 2024-08-16T19:26:17.420 INFO:teuthology.orchestra.run.smithi062.stdout: "restful", 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "snap_schedule" 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "available_modules": [ 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "alerts", 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.421 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.422 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How frequently to reexamine health status", 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.423 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.424 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.425 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.426 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.427 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.428 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.429 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.430 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.431 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_destination": { 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_destination", 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.432 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email address to send alerts to", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.433 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_from_name": { 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_from_name", 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.434 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "Ceph", 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Email From: name", 2024-08-16T19:26:17.435 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_host": { 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_host", 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.436 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.437 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP server", 2024-08-16T19:26:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.438 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_password": { 2024-08-16T19:26:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_password", 2024-08-16T19:26:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.439 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Password to authenticate with", 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.440 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_port": { 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_port", 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.441 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "465", 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP port", 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.442 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_sender": { 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_sender", 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.444 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SMTP envelope sender", 2024-08-16T19:26:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.445 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_ssl": { 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_ssl", 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.446 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-16T19:26:17.447 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "smtp_user": { 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "smtp_user", 2024-08-16T19:26:17.448 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.449 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "User to authenticate as", 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.450 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "balancer", 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.451 INFO:teuthology.orchestra.run.smithi062.stdout: "active": { 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "active", 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically balance PGs across cluster", 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_time": { 2024-08-16T19:26:17.453 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_time", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0000", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.454 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "beginning time of day to automatically balance", 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "begin_weekday": { 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "begin_weekday", 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:17.455 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:17.456 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_max_iterations": { 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_max_iterations", 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.457 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "25", 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "250", 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.458 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_metrics": { 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_metrics", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "pgs,objects,bytes", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.459 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_compat_step": { 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crush_compat_step", 2024-08-16T19:26:17.460 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.5", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0.001", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "0.999", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "aggressiveness of optimization", 2024-08-16T19:26:17.461 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "end_time": { 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_time", 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.462 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2359", 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ending time of day to automatically balance", 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.463 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "end_weekday": { 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "end_weekday", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "0", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "6", 2024-08-16T19:26:17.464 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.465 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.466 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.467 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.468 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.469 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.470 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.472 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.473 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.474 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.475 INFO:teuthology.orchestra.run.smithi062.stdout: "min_score": { 2024-08-16T19:26:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_score", 2024-08-16T19:26:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.476 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.477 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.478 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "upmap", 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.479 INFO:teuthology.orchestra.run.smithi062.stdout: "crush-compat", 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "none", 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "read", 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap", 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap-read" 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Balancer mode", 2024-08-16T19:26:17.480 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_ids": { 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_ids", 2024-08-16T19:26:17.481 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.482 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.483 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.484 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.485 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_deviation": { 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_deviation", 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:17.486 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.487 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "upmap_max_optimizations": { 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "upmap_max_optimizations", 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.488 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.489 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.490 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm", 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_down_multiplier": { 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_down_multiplier", 2024-08-16T19:26:17.491 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.492 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-16T19:26:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.493 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_refresh_rate": { 2024-08-16T19:26:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_refresh_rate", 2024-08-16T19:26:17.494 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "20", 2024-08-16T19:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: "agent_starting_port": { 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "agent_starting_port", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4721", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.497 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_ptrace": { 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_ptrace", 2024-08-16T19:26:17.498 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-16T19:26:17.499 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_interval": { 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_interval", 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.500 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.501 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "autotune_memory_target_ratio": { 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "autotune_memory_target_ratio", 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0.7", 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.502 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_log_destination": { 2024-08-16T19:26:17.503 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephadm_log_destination", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.504 INFO:teuthology.orchestra.run.smithi062.stdout: "file", 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "file,syslog", 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "syslog" 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.505 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "cgroups_split": { 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cgroups_split", 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.506 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.507 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "config_checks_enabled": { 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_checks_enabled", 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.508 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.509 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "config_dashboard": { 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_dashboard", 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.510 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.511 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_alertmanager": { 2024-08-16T19:26:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_alertmanager", 2024-08-16T19:26:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.512 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:17.513 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_base": { 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_base", 2024-08-16T19:26:17.514 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-16T19:26:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.515 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Container image name, without the tag", 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.516 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_elasticsearch": { 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_elasticsearch", 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.517 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-16T19:26:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.518 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "elasticsearch container image", 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_grafana": { 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_grafana", 2024-08-16T19:26:17.519 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.520 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_haproxy": { 2024-08-16T19:26:17.521 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_haproxy", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.522 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "HAproxy container image", 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_agent": { 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_agent", 2024-08-16T19:26:17.523 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-16T19:26:17.524 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger agent container image", 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.525 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_collector": { 2024-08-16T19:26:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_collector", 2024-08-16T19:26:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.526 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger collector container image", 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_jaeger_query": { 2024-08-16T19:26:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_jaeger_query", 2024-08-16T19:26:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.528 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-16T19:26:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.529 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Jaeger query container image", 2024-08-16T19:26:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.530 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_keepalived": { 2024-08-16T19:26:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_keepalived", 2024-08-16T19:26:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.531 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-16T19:26:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.532 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Keepalived container image", 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.533 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_loki": { 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_loki", 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.534 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Loki container image", 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_node_exporter": { 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_node_exporter", 2024-08-16T19:26:17.535 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:17.536 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_nvmeof": { 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_nvmeof", 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Nvme-of container image", 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.538 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_prometheus": { 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_prometheus", 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.539 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Prometheus container image", 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.540 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_promtail": { 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_promtail", 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-16T19:26:17.541 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Promtail container image", 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.542 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_samba": { 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_samba", 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.543 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Samba/SMB container image", 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image_snmp_gateway": { 2024-08-16T19:26:17.544 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_image_snmp_gateway", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "SNMP Gateway container image", 2024-08-16T19:26:17.545 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "container_init": { 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "container_init", 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.546 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Run podman/docker with `--init`", 2024-08-16T19:26:17.547 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "daemon_cache_timeout": { 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "daemon_cache_timeout", 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.548 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.549 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "default_cephadm_command_timeout": { 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_cephadm_command_timeout", 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.550 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "900", 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.551 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "default_registry": { 2024-08-16T19:26:17.552 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "default_registry", 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "docker.io", 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.553 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.554 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "device_cache_timeout": { 2024-08-16T19:26:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_cache_timeout", 2024-08-16T19:26:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.555 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1800", 2024-08-16T19:26:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.556 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache device inventory", 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "device_enhanced_scan": { 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_enhanced_scan", 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.557 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-16T19:26:17.558 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.559 INFO:teuthology.orchestra.run.smithi062.stdout: "facts_cache_timeout": { 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "facts_cache_timeout", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.560 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "seconds to cache host facts data", 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "grafana_dashboards_path": { 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "grafana_dashboards_path", 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.561 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-16T19:26:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.562 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "host_check_interval": { 2024-08-16T19:26:17.563 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "host_check_interval", 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.564 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to perform a host check", 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "hw_monitoring": { 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hw_monitoring", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "inventory_list_all": { 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "inventory_list_all", 2024-08-16T19:26:17.567 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.568 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.569 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.570 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.570 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.570 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.570 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.570 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.571 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.571 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.571 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.571 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.571 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.572 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.572 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.572 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.572 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.573 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.573 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.573 INFO:teuthology.orchestra.run.smithi062.stdout: "log_refresh_metadata": { 2024-08-16T19:26:17.573 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_refresh_metadata", 2024-08-16T19:26:17.574 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.574 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.574 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.574 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.575 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.576 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.577 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.578 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.579 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.581 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-16T19:26:17.582 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.583 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-16T19:26:17.584 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "*", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.585 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max_count_per_host": { 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_count_per_host", 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.587 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of daemons per service per host", 2024-08-16T19:26:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.588 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max_osd_draining_count": { 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_osd_draining_count", 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.589 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.590 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "migration_current": { 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "migration_current", 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.591 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "internal - do not modify", 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.592 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": { 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mode", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "root", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.593 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm-package", 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "root" 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "mode for remote execution of cephadm", 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.594 INFO:teuthology.orchestra.run.smithi062.stdout: "oob_default_addr": { 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "oob_default_addr", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "169.254.1.1", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.595 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "prometheus_alerts_path": { 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus_alerts_path", 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.596 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.597 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_insecure": { 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_insecure", 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.598 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.599 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_password": { 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_password", 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.600 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.601 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_url": { 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_url", 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.602 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "registry_username": { 2024-08-16T19:26:17.603 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "registry_username", 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.604 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "secure_monitoring_stack": { 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "secure_monitoring_stack", 2024-08-16T19:26:17.605 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.606 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "service_discovery_port": { 2024-08-16T19:26:17.607 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "service_discovery_port", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8765", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.608 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "cephadm service discovery port", 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.609 INFO:teuthology.orchestra.run.smithi062.stdout: "ssh_config_file": { 2024-08-16T19:26:17.610 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssh_config_file", 2024-08-16T19:26:17.610 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.610 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.610 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.610 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.611 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.612 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.612 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.612 INFO:teuthology.orchestra.run.smithi062.stdout: "use_agent": { 2024-08-16T19:26:17.612 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_agent", 2024-08-16T19:26:17.612 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-16T19:26:17.613 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "use_repo_digest": { 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "use_repo_digest", 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.614 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-16T19:26:17.615 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_failed_host_check": { 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_failed_host_check", 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.616 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if the host check fails", 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.617 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_daemons": { 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_daemons", 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.618 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_on_stray_hosts": { 2024-08-16T19:26:17.619 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_on_stray_hosts", 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.620 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.621 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.622 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.622 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.622 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crash", 2024-08-16T19:26:17.622 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.622 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.623 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.623 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.623 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.623 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.624 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.624 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.624 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.624 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.625 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.626 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.627 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.628 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.629 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.630 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.631 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "retain_interval": { 2024-08-16T19:26:17.632 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retain_interval", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "31536000", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.633 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_recent_interval": { 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_recent_interval", 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.634 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1209600", 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.635 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-16T19:26:17.636 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.636 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.636 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.636 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.636 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dashboard", 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.637 INFO:teuthology.orchestra.run.smithi062.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-16T19:26:17.638 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-16T19:26:17.638 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.638 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.638 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.638 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.639 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_HOST": { 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.640 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.641 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.642 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.643 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_ENABLED": { 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_ENABLED", 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.644 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.646 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.647 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "ENABLE_BROWSABLE_API": { 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.648 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.650 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.651 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.652 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-16T19:26:17.652 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-16T19:26:17.652 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.652 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.653 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.654 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.654 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.654 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.655 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-16T19:26:17.655 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-16T19:26:17.655 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.655 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.655 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.656 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.657 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.659 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.659 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.659 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.659 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-16T19:26:17.659 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.660 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.661 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.662 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.663 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.664 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.665 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_PASSWORD": { 2024-08-16T19:26:17.666 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.667 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.668 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.669 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_URL": { 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_URL", 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.670 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_API_USERNAME": { 2024-08-16T19:26:17.671 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.672 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.674 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.675 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-16T19:26:17.676 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.677 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.678 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.679 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_HOST": { 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.680 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.681 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.682 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.683 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.684 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.685 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.685 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.685 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.685 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.685 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-16T19:26:17.686 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.687 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.687 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.687 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.687 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.687 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.688 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-16T19:26:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-16T19:26:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.689 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.690 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.691 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.692 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.693 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.693 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.693 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.693 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-16T19:26:17.693 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.694 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.695 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.696 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.697 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.697 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.697 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.697 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.697 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-16T19:26:17.698 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.699 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_ENABLED": { 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.700 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.702 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-16T19:26:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.704 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.705 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8", 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "45", 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.707 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ACCESS_KEY": { 2024-08-16T19:26:17.708 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.709 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.710 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "admin", 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.711 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SECRET_KEY": { 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.712 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "RGW_API_SSL_VERIFY": { 2024-08-16T19:26:17.713 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.714 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.715 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.716 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.717 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.718 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:17.719 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.720 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: "cross_origin_url": { 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cross_origin_url", 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.721 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.722 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: "crt_file": { 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "crt_file", 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.723 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.724 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: "debug": { 2024-08-16T19:26:17.725 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "debug", 2024-08-16T19:26:17.726 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.726 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.726 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.726 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.726 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Enable/disable debug options", 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.727 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.728 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.728 INFO:teuthology.orchestra.run.smithi062.stdout: "jwt_token_ttl": { 2024-08-16T19:26:17.728 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "jwt_token_ttl", 2024-08-16T19:26:17.728 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.728 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "28800", 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.729 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:17.730 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.731 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.732 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.733 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.734 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.735 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.736 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.736 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.736 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.736 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.737 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.738 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.739 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.740 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "motd": { 2024-08-16T19:26:17.741 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "motd", 2024-08-16T19:26:17.742 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.742 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.742 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.742 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.742 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "The message of the day", 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.743 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect_resolve_ip_addr": { 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "redirect_resolve_ip_addr", 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.744 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.745 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.746 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8080", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.748 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:17.749 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.750 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl_server_port": { 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl_server_port", 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.751 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8443", 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.752 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "redirect", 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.753 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "redirect" 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.754 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.755 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.756 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "url_prefix": { 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url_prefix", 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.757 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.758 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.759 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_monitoring": { 2024-08-16T19:26:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_monitoring", 2024-08-16T19:26:17.760 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.761 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "monitor device health metrics", 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.762 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.763 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.763 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.763 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.763 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.764 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.765 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.766 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.766 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.766 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.767 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.767 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.767 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.768 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.768 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.768 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.768 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.768 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.769 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.769 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.769 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.770 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.771 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.771 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.771 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.771 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.771 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.772 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.772 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.772 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.772 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.772 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.773 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.773 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.773 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.773 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.773 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.774 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.774 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.774 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.774 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.774 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.775 INFO:teuthology.orchestra.run.smithi062.stdout: "mark_out_threshold": { 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mark_out_threshold", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "2419200", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.776 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "pool_name": { 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pool_name", 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.777 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "device_health_metrics", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.778 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "retention_period": { 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "retention_period", 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.779 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15552000", 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long to retain device health metrics", 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_frequency": { 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_frequency", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.781 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "self_heal": { 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "self_heal", 2024-08-16T19:26:17.782 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-16T19:26:17.783 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.784 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how frequently to wake up and check device health", 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "warn_threshold": { 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "warn_threshold", 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7257600", 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.786 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.787 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "diskprediction_local", 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.788 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.789 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.790 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.791 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.793 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.794 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.795 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.796 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.796 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.796 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.796 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.797 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: "predict_interval": { 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predict_interval", 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.798 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.799 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.799 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "86400", 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.800 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.801 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.801 INFO:teuthology.orchestra.run.smithi062.stdout: "predictor_model": { 2024-08-16T19:26:17.801 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "predictor_model", 2024-08-16T19:26:17.802 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.802 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.802 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.802 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "prophetstor", 2024-08-16T19:26:17.802 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.803 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.804 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.804 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.804 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.804 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.804 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "600", 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.805 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.806 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.806 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.806 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "influx", 2024-08-16T19:26:17.806 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": false, 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "influxdb python module not found", 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "batch_size": { 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "batch_size", 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.807 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5000", 2024-08-16T19:26:17.808 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.808 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.808 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.808 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "database": { 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "database", 2024-08-16T19:26:17.809 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.810 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.810 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.810 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "ceph", 2024-08-16T19:26:17.810 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.810 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.811 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.811 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-16T19:26:17.811 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.811 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.811 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "hostname": { 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "hostname", 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.812 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server hostname", 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "30", 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "5", 2024-08-16T19:26:17.814 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.815 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.815 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-16T19:26:17.815 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.815 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.816 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.817 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.818 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.819 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.820 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.821 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.822 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.823 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "password": { 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "password", 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.824 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.825 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "password of InfluxDB server user", 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: "port": { 2024-08-16T19:26:17.826 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "port", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "8086", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.827 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "InfluxDB server port", 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "ssl": { 2024-08-16T19:26:17.828 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ssl", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "false", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.829 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "threads": { 2024-08-16T19:26:17.830 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threads", 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:17.831 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "32", 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.832 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "username": { 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "username", 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.833 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "username of InfluxDB server user", 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.834 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "verify_ssl": { 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "verify_ssl", 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.835 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "true", 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.836 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "insights", 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.837 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.838 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.838 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.838 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.838 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.838 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.839 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.840 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.840 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.840 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.840 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.840 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.841 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.842 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.843 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.844 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.845 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.845 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.845 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.845 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.845 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.846 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.847 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.848 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.849 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.849 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "iostat", 2024-08-16T19:26:17.849 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.849 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.850 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.851 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.851 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.851 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.851 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.851 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.852 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.852 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.852 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.852 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.852 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.853 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.853 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.853 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.853 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.854 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.854 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.854 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.855 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.855 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.855 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.855 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.855 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.856 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.857 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.858 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.859 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.860 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.861 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "k8sevents", 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_event_retention_days": { 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "ceph_event_retention_days", 2024-08-16T19:26:17.862 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "7", 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.863 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-16T19:26:17.864 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.864 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.864 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.864 INFO:teuthology.orchestra.run.smithi062.stdout: "config_check_secs": { 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "config_check_secs", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "10", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.865 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.866 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-16T19:26:17.866 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.866 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.866 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.867 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.868 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.868 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.868 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.868 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.869 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.869 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.869 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.869 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.869 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.870 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.870 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.870 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.870 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.871 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.871 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.871 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.871 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.872 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.872 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.872 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.872 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.872 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.873 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.873 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.873 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.873 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.874 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.875 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.876 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.877 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "localpool", 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.878 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "failure_domain": { 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "failure_domain", 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "host", 2024-08-16T19:26:17.879 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "failure domain for any created local pool", 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.880 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.881 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.882 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.882 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.882 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.882 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.882 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.883 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.884 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.885 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.887 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.888 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.889 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "min_size": { 2024-08-16T19:26:17.890 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "min_size", 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.891 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default min_size for any created local pool", 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.892 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.893 INFO:teuthology.orchestra.run.smithi062.stdout: "num_rep": { 2024-08-16T19:26:17.893 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "num_rep", 2024-08-16T19:26:17.893 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.893 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.893 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3", 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default replica count for any created local pool", 2024-08-16T19:26:17.894 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_num": { 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_num", 2024-08-16T19:26:17.895 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "128", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "default pg_num for any created local pool", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.896 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "prefix": { 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prefix", 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.897 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "name prefix for any created local pool", 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.898 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: "subtree": { 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "subtree", 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.899 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "rack", 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-16T19:26:17.900 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.901 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mds_autoscaler", 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.902 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.903 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.904 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.905 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.905 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.905 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.905 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.905 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.906 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.907 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.908 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.909 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.910 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.911 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.912 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirroring", 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.913 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.914 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.915 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.916 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.917 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.918 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.919 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.920 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.921 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "nfs", 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.922 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.923 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.924 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.925 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.926 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.927 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.928 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.929 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.930 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.931 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.931 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.931 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.931 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.931 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:17.932 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "fail_fs": { 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "fail_fs", 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.936 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.937 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.937 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.937 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.937 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.937 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.938 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.939 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.939 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.939 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.939 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.939 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.940 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.940 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.940 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.940 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.940 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.941 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.941 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.941 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.941 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.941 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.942 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.943 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.944 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.945 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator": { 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "orchestrator", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.947 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm", 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "rook", 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "test_orchestrator" 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Orchestrator backend", 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.948 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_perf_query", 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.951 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.952 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.953 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.954 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.955 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.956 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.957 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "osd_support", 2024-08-16T19:26:17.958 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.959 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.960 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.961 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.962 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.964 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.965 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.966 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.967 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.968 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "pg_autoscaler", 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.969 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.970 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.970 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.970 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.970 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.970 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.971 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.972 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.973 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.974 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.975 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.975 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.975 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.975 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.975 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.976 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.978 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.979 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.980 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "threshold": { 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "threshold", 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.981 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "3.0", 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1.0", 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "scaling threshold", 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.982 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "progress", 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:17.983 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_pg_recovery_event": { 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_pg_recovery_event", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.984 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:17.985 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.986 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.987 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.988 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.989 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.990 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:17.991 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:17.992 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.993 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.994 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "max_completed_events": { 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_completed_events", 2024-08-16T19:26:17.995 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "50", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "number of past completed events to remember", 2024-08-16T19:26:17.996 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: "sleep_interval": { 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "sleep_interval", 2024-08-16T19:26:17.997 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "5", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "how long the module is going to sleep", 2024-08-16T19:26:17.998 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:17.999 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "prometheus", 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "cache": { 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cache", 2024-08-16T19:26:18.000 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.001 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "exclude_perf_counters": { 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "exclude_perf_counters", 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.002 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.003 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.004 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.005 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.007 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.009 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.010 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.012 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.013 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools": { 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools", 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.014 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.015 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "300", 2024-08-16T19:26:18.016 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrape_interval": { 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "scrape_interval", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15.0", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.018 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:18.019 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "::", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-16T19:26:18.020 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.021 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "9283", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.022 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "stale_cache_strategy": { 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stale_cache_strategy", 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "log", 2024-08-16T19:26:18.023 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.024 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_behaviour": { 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_behaviour", 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "default", 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.025 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "default", 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "error" 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.026 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "standby_error_status_code": { 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "standby_error_status_code", 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "500", 2024-08-16T19:26:18.027 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "400", 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "599", 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.028 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.029 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.030 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.031 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.032 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.033 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.034 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.035 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.036 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.037 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.038 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_snap_create": { 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_snap_create", 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10", 2024-08-16T19:26:18.039 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mirror_snapshot_schedule": { 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mirror_snapshot_schedule", 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.041 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.042 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.042 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.042 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.042 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.042 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: "trash_purge_schedule": { 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "trash_purge_schedule", 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.043 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.044 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.044 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.044 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.044 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.045 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.045 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.045 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.045 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.045 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "restful", 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.046 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_auth": { 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enable_auth", 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.047 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.048 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "key_file": { 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "key_file", 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.049 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.050 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.051 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.052 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.053 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.054 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.055 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.055 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.055 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.056 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.057 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.058 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.059 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.060 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.061 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.062 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.062 INFO:teuthology.orchestra.run.smithi062.stdout: "server_addr": { 2024-08-16T19:26:18.062 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_addr", 2024-08-16T19:26:18.062 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.062 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.063 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.063 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.063 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.063 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.063 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "server_port": { 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "server_port", 2024-08-16T19:26:18.064 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.065 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.066 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rgw", 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.067 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.068 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.071 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.072 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.073 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.074 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.075 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rook", 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.076 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.077 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.078 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.079 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.080 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.081 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.082 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.083 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.084 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.085 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: "storage_class": { 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "storage_class", 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.086 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "local", 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.087 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "selftest", 2024-08-16T19:26:18.088 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.089 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.090 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.091 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.092 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.093 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.095 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.096 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.097 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.098 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "roption1": { 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption1", 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.099 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.100 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "roption2": { 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "roption2", 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "xyz", 2024-08-16T19:26:18.101 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.102 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption1": { 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption1", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption2": { 2024-08-16T19:26:18.104 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption2", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.105 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption3": { 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption3", 2024-08-16T19:26:18.106 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "float", 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.107 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption4": { 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption4", 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.108 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.109 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption5": { 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption5", 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.110 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.111 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption6": { 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption6", 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.112 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "rwoption7": { 2024-08-16T19:26:18.113 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rwoption7", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "1", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "42", 2024-08-16T19:26:18.114 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.115 INFO:teuthology.orchestra.run.smithi062.stdout: "testkey": { 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testkey", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.116 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.117 INFO:teuthology.orchestra.run.smithi062.stdout: "testlkey": { 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testlkey", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.118 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "testnewline": { 2024-08-16T19:26:18.119 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "testnewline", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.120 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.121 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "allow_m_granularity": { 2024-08-16T19:26:18.122 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "allow_m_granularity", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.123 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "allow minute scheduled snapshots", 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: "dump_on_update": { 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "dump_on_update", 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.125 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "dump database to debug log on update", 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.126 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.127 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.128 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.129 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.130 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.131 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.132 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.133 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.134 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.135 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "stats", 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.136 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.137 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.138 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.139 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.140 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.141 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.142 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.143 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.144 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "status", 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.145 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.146 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.147 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.148 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.149 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.150 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.151 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.151 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.151 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.151 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.151 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.152 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.153 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.154 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telegraf", 2024-08-16T19:26:18.155 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "address": { 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "address", 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.156 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.157 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:18.158 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.159 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.159 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "15", 2024-08-16T19:26:18.159 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.159 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.159 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.160 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.161 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.161 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.161 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.161 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.161 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.162 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.162 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.162 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.162 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.163 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.164 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.164 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.164 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.164 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.164 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.165 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.165 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.165 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.165 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.166 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.167 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.168 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.169 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.170 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "telemetry", 2024-08-16T19:26:18.171 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_basic": { 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_basic", 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.172 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.173 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_crash": { 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_crash", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.174 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_device": { 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_device", 2024-08-16T19:26:18.175 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.176 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_ident": { 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_ident", 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.177 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.178 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "channel_perf": { 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "channel_perf", 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.179 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.180 INFO:teuthology.orchestra.run.smithi062.stdout: "contact": { 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "contact", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.181 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "description": { 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "description", 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.182 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.183 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "device_url": { 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "device_url", 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.184 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.185 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "enabled": { 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "enabled", 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.186 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:18.187 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "24", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "8", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.188 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: "last_opt_revision": { 2024-08-16T19:26:18.189 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "last_opt_revision", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "1", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.190 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard": { 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard", 2024-08-16T19:26:18.191 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.192 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "leaderboard_description": { 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "leaderboard_description", 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.193 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.194 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.195 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.196 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.197 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.198 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.199 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.200 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.201 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.202 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.203 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.204 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "organization": { 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "organization", 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.205 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "proxy": { 2024-08-16T19:26:18.206 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "proxy", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "url": { 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "url", 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.208 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.209 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "test_orchestrator", 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.210 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.211 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.212 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.213 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.214 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.215 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.216 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.217 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.218 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.219 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.220 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.221 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.221 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.221 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.221 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.221 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.222 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.223 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.223 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.223 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.223 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.225 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.225 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.225 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.225 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.225 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.226 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.227 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.228 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.229 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "max_concurrent_clones": { 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "max_concurrent_clones", 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.230 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.231 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "4", 2024-08-16T19:26:18.231 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.231 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.231 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.231 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "periodic_async_work": { 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "periodic_async_work", 2024-08-16T19:26:18.232 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.233 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.233 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.233 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.233 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.234 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.234 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Periodically check for async work", 2024-08-16T19:26:18.234 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.234 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.235 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.235 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.235 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_delay": { 2024-08-16T19:26:18.235 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_delay", 2024-08-16T19:26:18.235 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.236 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.236 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.236 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "0", 2024-08-16T19:26:18.236 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.237 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.237 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.237 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-16T19:26:18.237 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.237 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "snapshot_clone_no_wait": { 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snapshot_clone_no_wait", 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.238 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "True", 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.239 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix", 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "can_run": true, 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "error_string": "", 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "module_options": { 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "discovery_interval": { 2024-08-16T19:26:18.240 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "discovery_interval", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "uint", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "100", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.241 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "identifier": { 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "identifier", 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.242 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.243 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "interval": { 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "interval", 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "secs", 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "60", 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.244 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "log_level": { 2024-08-16T19:26:18.245 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_level", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.246 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.247 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster": { 2024-08-16T19:26:18.248 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.249 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_cluster_level": { 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_cluster_level", 2024-08-16T19:26:18.250 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "info", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [ 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "critical", 2024-08-16T19:26:18.251 INFO:teuthology.orchestra.run.smithi062.stdout: "debug", 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "error", 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "info", 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "warning" 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.252 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": { 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "log_to_file", 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "bool", 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 1, 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "False", 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.253 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.254 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_host": { 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_host", 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "", 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.255 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.256 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_port": { 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_port", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "int", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "10051", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.257 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.258 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.258 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.258 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.258 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "zabbix_sender": { 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "zabbix_sender", 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "str", 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "level": "advanced", 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "flags": 0, 2024-08-16T19:26:18.259 INFO:teuthology.orchestra.run.smithi062.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "min": "", 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "max": "", 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "enum_allowed": [], 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "desc": "", 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "long_desc": "", 2024-08-16T19:26:18.260 INFO:teuthology.orchestra.run.smithi062.stdout: "tags": [], 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: "see_also": [] 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.261 INFO:teuthology.orchestra.run.smithi062.stdout: "services": { 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "dashboard": "https://172.21.15.62:8443/" 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "always_on_modules": { 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "octopus": [ 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-08-16T19:26:18.262 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-08-16T19:26:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "pacific": [ 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-08-16T19:26:18.264 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-08-16T19:26:18.265 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "quincy": [ 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-08-16T19:26:18.266 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.267 INFO:teuthology.orchestra.run.smithi062.stdout: "reef": [ 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-08-16T19:26:18.268 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: ], 2024-08-16T19:26:18.269 INFO:teuthology.orchestra.run.smithi062.stdout: "squid": [ 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "balancer", 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "crash", 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "devicehealth", 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "orchestrator", 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "pg_autoscaler", 2024-08-16T19:26:18.270 INFO:teuthology.orchestra.run.smithi062.stdout: "progress", 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: "rbd_support", 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: "status", 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: "telemetry", 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: "volumes" 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.271 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: "last_failure_osd_epoch": 93, 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: "active_clients": [ 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "devicehealth", 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.272 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2115648217 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.273 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "libcephsqlite", 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2837530432 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.274 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "rbd_support", 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.275 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 821032042 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.276 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 918461861 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.277 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "snap_schedule", 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 569225520 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.278 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "volumes", 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: "addrvec": [ 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: "type": "v2", 2024-08-16T19:26:18.279 INFO:teuthology.orchestra.run.smithi062.stdout: "addr": "172.21.15.62:0", 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout: "nonce": 2839752767 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-08-16T19:26:18.280 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:18.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:15 smithi161 ceph-mon[37576]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-08-16T19:26:18.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:15 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/910918546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:15 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/6466589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:15 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1673856355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-16T19:26:18.282 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:15 smithi155 ceph-mon[37413]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-08-16T19:26:18.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:15 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/910918546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:15 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/6466589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.283 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:15 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1673856355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-16T19:26:18.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 msgr2=0x7fde580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 0x7fde580a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde700fe8f0 tx=0x7fde64008040 comp rx=0 tx=0).stop 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 msgr2=0x7fde70197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fde60007b50 tx=0x7fde6000e420 comp rx=0 tx=0).stop 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 shutdown_connections 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fde580a2d00 0x7fde580a51c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde70106700 0x7fde7019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde70101090 0x7fde70197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:18.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 --2- 172.21.15.62:0/1673856355 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde701006f0 0x7fde70197010 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:18.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 >> 172.21.15.62:0/1673856355 conn(0x7fde700fc3d0 msgr2=0x7fde700fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:18.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 shutdown_connections 2024-08-16T19:26:18.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:15.687+0000 7fde76ed3640 1 -- 172.21.15.62:0/1673856355 wait complete. 2024-08-16T19:26:18.291 DEBUG:tasks.check_counter:Getting stats from a 2024-08-16T19:26:18.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-16T19:26:18.292 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:15 smithi062 ceph-mon[32993]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 168 op/s 2024-08-16T19:26:18.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:15 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/910918546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:15 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/6466589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-16T19:26:18.293 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:15 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1673856355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-16T19:26:18.294 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:17 smithi062 ceph-mon[32993]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:18.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:17 smithi155 ceph-mon[37413]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:18.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:17 smithi161 ceph-mon[37576]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:18.836 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:19.968 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:19.968 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1696, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1698, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 17295209, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 539329, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.170224458, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.051766370, 2024-08-16T19:26:19.969 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.053585537, 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.049740520, 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1696, 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.060835624, 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000035870 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:19.970 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:19.971 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1401, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1626, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 50684505, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 1087112, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 9, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.161468442, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.041929800, 2024-08-16T19:26:19.972 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.070659439, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.037023724, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1623, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.056603916, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000034876 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.973 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 642480 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 931, 2024-08-16T19:26:19.974 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 871, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 14553818, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 360996, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.107114439, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.036481204, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.067908558, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.012152120, 2024-08-16T19:26:19.975 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 871, 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.038091605, 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000043733 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1832528, 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 188640 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.977 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6147, 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.148564751, 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000024168 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:19.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.979 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.980 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-08-16T19:26:19.981 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 5986, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 18, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 5968, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 5961, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-08-16T19:26:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 5, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-08-16T19:26:19.983 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:19.984 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:19.985 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-08-16T19:26:19.986 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-08-16T19:26:19.987 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:19.988 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:19.989 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:19.990 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-08-16T19:26:19.991 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 37540, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-08-16T19:26:19.992 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 50, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 5, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-08-16T19:26:19.993 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 83885962, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 83878672, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.994 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 37540 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 782, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 6193, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 5411, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 41, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 49, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 8, 2024-08-16T19:26:19.995 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 6013, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 6378, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 365, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 61712, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215328 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.996 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.997 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:19.998 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:19.999 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.000 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-08-16T19:26:20.001 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.002 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.004 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-08-16T19:26:20.004 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.006 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-08-16T19:26:20.007 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.010 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.011 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.012 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.013 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.014 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:20.015 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-08-16T19:26:20.016 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 4, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 3, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 2524, 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:20.017 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:20.019 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:20.020 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-08-16T19:26:20.021 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 253979, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 882, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:20.022 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 196, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:20.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 6222431, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 78825, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-08-16T19:26:20.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3636, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3636, 2024-08-16T19:26:20.025 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3636, 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.501959264, 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000688107 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3636, 2024-08-16T19:26:20.026 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3637 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3636, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 3636, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-08-16T19:26:20.027 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 2045, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 1589, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-08-16T19:26:20.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-08-16T19:26:20.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-08-16T19:26:20.030 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-08-16T19:26:20.031 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-08-16T19:26:20.032 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 113, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 25, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 9, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 9, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-08-16T19:26:20.033 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-08-16T19:26:20.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-08-16T19:26:20.035 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:20.036 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 4024, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 82148799, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 4024, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4024, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 82148799, 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:20.037 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:20.038 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3636, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 79692158, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3636, 2024-08-16T19:26:20.039 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 79692158, 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:20.040 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3636, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3636, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3636, 2024-08-16T19:26:20.041 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3636, 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:20.042 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:20.043 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x55e9ce4b6b40": { 2024-08-16T19:26:20.044 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:20.045 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.046 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:20.047 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:20.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:19 smithi062 ceph-mon[32993]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:20.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:19 smithi155 ceph-mon[37413]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:20.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:19 smithi161 ceph-mon[37576]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 162 op/s 2024-08-16T19:26:20.831 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-08-16T19:26:20.831 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-08-16T19:26:20.832 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-08-16T19:26:20.832 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-08-16T19:26:20.832 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-08-16T19:26:20.832 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:20.832 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:20.832 DEBUG:tasks.check_counter:Getting stats from d 2024-08-16T19:26:20.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-16T19:26:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:22.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:21 smithi062 ceph-mon[32993]: pgmap v499: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 9.9 MiB/s wr, 247 op/s 2024-08-16T19:26:22.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:21 smithi155 ceph-mon[37413]: pgmap v499: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 9.9 MiB/s wr, 247 op/s 2024-08-16T19:26:22.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:21 smithi161 ceph-mon[37576]: pgmap v499: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 9.9 MiB/s wr, 247 op/s 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4, 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3024, 2024-08-16T19:26:22.474 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 448, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.003344467, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000039328, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002312572, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:22.475 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.004268192, 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001422730 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:22.476 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 231, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 480, 2024-08-16T19:26:22.477 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 275808, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.019092954, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.014033194, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002192613, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:22.478 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 228, 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.012923495, 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000056681 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:22.479 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 275808 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 312, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 302, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1469024, 2024-08-16T19:26:22.480 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 186160, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.048962370, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.019248395, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.041327324, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 301, 2024-08-16T19:26:22.481 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.015276289, 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000050751 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:22.482 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1469024, 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 186160 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-08-16T19:26:22.483 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:22.484 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:22.485 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:22.486 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:22.487 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-08-16T19:26:22.488 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-08-16T19:26:22.489 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:22.489 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:22.489 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-08-16T19:26:22.490 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:22.491 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 315, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1425011, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 315, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:22.492 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 315, 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1425011, 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:22.493 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:23.219 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-16T19:26:23.219 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-16T19:26:23.220 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-08-16T19:26:23.220 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-08-16T19:26:23.220 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-16T19:26:23.220 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-16T19:26:23.220 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-16T19:26:23.220 DEBUG:tasks.check_counter:Getting stats from g 2024-08-16T19:26:23.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-16T19:26:23.519 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:24.000 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:24.000 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:24.000 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1885, 2024-08-16T19:26:24.000 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1827, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 14879306, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 665514, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.197732600, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.075725703, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.084128589, 2024-08-16T19:26:24.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.039118550, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1826, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.075127394, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000041143 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1829776, 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 188832 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1809, 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2032, 2024-08-16T19:26:24.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 26332442, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 1216021, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 9, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.159135922, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.043764599, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.062577440, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.041162791, 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:24.004 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2030, 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.066904263, 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000032957 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 642928 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 330, 2024-08-16T19:26:24.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 332, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 16880012, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 104952, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.040167918, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.006960042, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.021682632, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.009012175, 2024-08-16T19:26:24.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 330, 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.011342337, 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000034370 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.009 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6192, 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.133821525, 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000021612 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.010 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-08-16T19:26:24.011 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-08-16T19:26:24.012 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 4621, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 57, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 4564, 2024-08-16T19:26:24.013 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 4540, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 7, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-08-16T19:26:24.014 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-08-16T19:26:24.015 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-08-16T19:26:24.016 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:24.017 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-08-16T19:26:24.018 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-08-16T19:26:24.019 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-08-16T19:26:24.020 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-08-16T19:26:24.021 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-08-16T19:26:24.022 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 25812, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 35, 2024-08-16T19:26:24.023 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 4, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 59425114, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 59421511, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.024 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 25812 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 789, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 4517, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 3728, 2024-08-16T19:26:24.025 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 46, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 50, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 4, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 4625, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 4732, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 107, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-08-16T19:26:24.026 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 59628, 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215328 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-08-16T19:26:24.027 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.028 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-08-16T19:26:24.030 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-08-16T19:26:24.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.033 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.035 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-08-16T19:26:24.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.036 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.037 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-08-16T19:26:24.038 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.038 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.038 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.038 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.038 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-08-16T19:26:24.039 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.039 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.039 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.039 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:24.040 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.041 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.041 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.041 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.041 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:24.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.042 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.042 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-08-16T19:26:24.043 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.043 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.043 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.044 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:24.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.044 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.046 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-08-16T19:26:24.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.046 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.047 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:24.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.047 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-08-16T19:26:24.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.051 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-08-16T19:26:24.053 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-08-16T19:26:24.053 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 4, 2024-08-16T19:26:24.053 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 2, 2024-08-16T19:26:24.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 2, 2024-08-16T19:26:24.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 3056, 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:24.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:24.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:24.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:24.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:24.059 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 261902, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 1134, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:24.060 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:24.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 196, 2024-08-16T19:26:24.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:24.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:24.062 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:24.062 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-08-16T19:26:24.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 5835720, 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 77265, 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-08-16T19:26:24.063 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.064 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-08-16T19:26:24.064 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-08-16T19:26:24.064 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-08-16T19:26:24.064 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 3631, 2024-08-16T19:26:24.064 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 3631, 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3631, 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.306962440, 2024-08-16T19:26:24.065 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000635351 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3631, 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3632 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 3631, 2024-08-16T19:26:24.066 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 3631, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 2052, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 1577, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-08-16T19:26:24.067 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:24.068 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-08-16T19:26:24.069 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-08-16T19:26:24.070 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-08-16T19:26:24.071 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 113, 2024-08-16T19:26:24.072 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 25, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 9, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 10, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 1, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-08-16T19:26:24.073 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:24.074 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:24.075 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:24.076 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:24.077 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:24.077 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:24.077 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 4020, 2024-08-16T19:26:24.077 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 57707335, 2024-08-16T19:26:24.077 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 4020, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4020, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 57707335, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.078 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:24.079 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3631, 2024-08-16T19:26:24.080 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 55251710, 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3631, 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 55251710, 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.081 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:24.082 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 3631, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 3631, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 3631, 2024-08-16T19:26:24.083 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 3631, 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.084 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:24.085 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:24.086 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:24.086 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:24.086 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:24.087 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:24.087 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-08-16T19:26:24.087 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-08-16T19:26:24.087 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:24.088 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.088 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.088 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.088 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.088 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5617bd4aab40": { 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-08-16T19:26:24.089 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-08-16T19:26:24.090 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:24.090 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:24.090 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:24.090 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:24.090 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.091 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:24.092 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:24.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:23 smithi062 ceph-mon[32993]: pgmap v500: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 9.1 MiB/s wr, 225 op/s 2024-08-16T19:26:24.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:23 smithi155 ceph-mon[37413]: pgmap v500: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 9.1 MiB/s wr, 225 op/s 2024-08-16T19:26:24.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:23 smithi161 ceph-mon[37576]: pgmap v500: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 9.1 MiB/s wr, 225 op/s 2024-08-16T19:26:25.109 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-08-16T19:26:25.109 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-08-16T19:26:25.109 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-08-16T19:26:25.110 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-08-16T19:26:25.110 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-08-16T19:26:25.110 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:25.110 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:25.110 DEBUG:tasks.check_counter:Getting stats from j 2024-08-16T19:26:25.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-16T19:26:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:26.000 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-08-16T19:26:26.001 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001317823, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000314857, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:26.002 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.003 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 7, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 235, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3504, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 276608, 2024-08-16T19:26:26.004 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.019854396, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.012638102, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.004198796, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 233, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.025364434, 2024-08-16T19:26:26.005 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000108860 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 276608 2024-08-16T19:26:26.006 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 314, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 304, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1469568, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 187440, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-08-16T19:26:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.050015971, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.020813112, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.040277555, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 303, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.014900884, 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000049177 2024-08-16T19:26:26.008 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1469568, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 187440 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:26.010 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:26.011 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:26.012 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:26.013 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:26.014 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 147930, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 28, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-08-16T19:26:26.015 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-08-16T19:26:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:26.017 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-08-16T19:26:26.018 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 317, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1425271, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 317, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 317, 2024-08-16T19:26:26.019 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1425271, 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:26.020 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-08-16T19:26:26.021 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-08-16T19:26:26.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:25 smithi062 ceph-mon[32993]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 296 op/s 2024-08-16T19:26:26.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:25 smithi155 ceph-mon[37413]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 296 op/s 2024-08-16T19:26:26.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:25 smithi161 ceph-mon[37576]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 296 op/s 2024-08-16T19:26:26.790 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-16T19:26:26.790 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-16T19:26:26.790 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-08-16T19:26:26.790 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-08-16T19:26:26.790 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-16T19:26:26.791 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-16T19:26:26.791 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-16T19:26:26.791 DEBUG:tasks.check_counter:Getting stats from b 2024-08-16T19:26:26.791 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-16T19:26:27.896 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:26:28.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:27 smithi155 ceph-mon[37413]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:28.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:27 smithi161 ceph-mon[37576]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:28.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:27 smithi062 ceph-mon[32993]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:28.631 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-16T19:26:28.632 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:28.632 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 143401, 2024-08-16T19:26:28.632 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 197591, 2024-08-16T19:26:28.632 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 32589003, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 270079235, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 9.028182515, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 3.828843934, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 5.069349810, 2024-08-16T19:26:28.633 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.614430000, 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 197586, 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 4.758359581, 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000024082 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.634 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 293532, 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.048020062, 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000163 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 94608, 2024-08-16T19:26:28.635 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 469168 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 333899, 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 449462, 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 93910845, 2024-08-16T19:26:28.636 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 484951268, 2024-08-16T19:26:28.637 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2024-08-16T19:26:28.637 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-08-16T19:26:28.637 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 19.680286250, 2024-08-16T19:26:28.637 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 9.094464588, 2024-08-16T19:26:28.637 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 11.812549988, 2024-08-16T19:26:28.638 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.931890920, 2024-08-16T19:26:28.638 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:28.638 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 449460, 2024-08-16T19:26:28.638 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 12.927121734, 2024-08-16T19:26:28.638 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000028761 2024-08-16T19:26:28.639 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.639 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:28.639 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 678316, 2024-08-16T19:26:28.639 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.103408681, 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000152 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 497040, 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 296432 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.640 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 259216, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 369594, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 60152090, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 256193506, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 70, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 13.952792987, 2024-08-16T19:26:28.641 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 5.955265805, 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 8.469368378, 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.548602292, 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 369596, 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7.778565240, 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000021046 2024-08-16T19:26:28.642 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 589909, 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.084459746, 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000143 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1923296, 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 203568 2024-08-16T19:26:28.643 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:28.644 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 83404, 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 57.921844371, 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000694473 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:28.645 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 20810, 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.632473511, 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000174554 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-16T19:26:28.646 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 131253, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 117552, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 117552, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1064.150139800, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.009052590 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-16T19:26:28.647 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 13701, 2024-08-16T19:26:28.648 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 20, 2024-08-16T19:26:28.649 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 66, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 5, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 2, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 1008, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 59, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 480, 2024-08-16T19:26:28.650 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 469, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 923, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 2928, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 148, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 148, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 15, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 199796, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 136987, 2024-08-16T19:26:28.651 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 13670, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 133, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 752, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 243954, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 27, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 26, 2024-08-16T19:26:28.652 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 2252, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 5, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 159, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 728, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 211697321, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-08-16T19:26:28.653 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:28.654 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 20, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 34000, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 46789, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 53799, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 11179, 2024-08-16T19:26:28.655 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 5292, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 65581 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 457, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 356, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 215, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 333, 2024-08-16T19:26:28.656 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 3211, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 10846, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 10846, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:28.657 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 14, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 2592, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 32, 2024-08-16T19:26:28.658 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 7, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 86004, 2024-08-16T19:26:28.659 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 22501, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 12291, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 73713, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 10210, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 159, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 31, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 16, 2024-08-16T19:26:28.660 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 143, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 6, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 15, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 42521852, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 587667686, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-08-16T19:26:28.661 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 43759, 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 151.271913599, 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003456932 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.662 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 547, 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 13241, 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 12694, 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 144, 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 367, 2024-08-16T19:26:28.663 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 223, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 1026, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 26662, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 25636, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 148, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 13271, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 13123, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 94228, 2024-08-16T19:26:28.664 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 231712 2024-08-16T19:26:28.665 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.665 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-16T19:26:28.665 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 183399, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 46041, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 131253, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 1521, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 92078, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11590, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 293.817621528, 2024-08-16T19:26:28.666 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.025350959 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 30152, 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 157.920998682, 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005237496 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:28.667 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.668 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.669 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.669 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-16T19:26:28.669 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 25416, 2024-08-16T19:26:28.669 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 163.401641442, 2024-08-16T19:26:28.669 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006429085 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-16T19:26:28.670 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000181685, 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000181685 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.671 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.672 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 16, 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.005258051, 2024-08-16T19:26:28.673 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000328628 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 119, 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.811514111, 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006819446 2024-08-16T19:26:28.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 16, 2024-08-16T19:26:28.675 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.457751870, 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.028609491 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10, 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.023477775, 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002347777 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.676 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 37081, 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 130.799805882, 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003527407 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1971, 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 49.960821907, 2024-08-16T19:26:28.677 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.025347956 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.678 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001308985, 2024-08-16T19:26:28.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001308985 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10, 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.102929145, 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.010292914 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.680 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 44, 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 64.878910744, 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 1.474520698 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.681 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 271, 2024-08-16T19:26:28.682 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.291005830, 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004763859 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.683 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8, 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.020489134, 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002561141 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.684 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10846, 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 200.656423029, 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.018500500 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.685 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.686 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 6, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 4, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-16T19:26:28.687 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 23997, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 11998, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 941, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 240, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 240, 2024-08-16T19:26:28.688 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:28.689 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:28.690 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:28.691 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:28.692 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 397362, 2024-08-16T19:26:28.693 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 585, 2024-08-16T19:26:28.693 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 176, 2024-08-16T19:26:28.693 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 2, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:28.694 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 196, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 2787149, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 27407, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:28.695 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 68009, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 586524403, 2024-08-16T19:26:28.696 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 37, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 68009, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 68009, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 264.326483579, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003886639 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-16T19:26:28.697 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 68009, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 68933 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 68009, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 465, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 67544, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-16T19:26:28.698 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 51, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 445, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 44204, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 290, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-16T19:26:28.699 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 5, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 22368, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 98, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:28.700 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-16T19:26:28.701 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-16T19:26:28.702 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-16T19:26:28.703 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 113, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 31, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-16T19:26:28.704 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 1167, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 40, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 265 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-16T19:26:28.705 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 1897 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 10, 2024-08-16T19:26:28.706 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 5, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 21692, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 10846, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:28.707 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 736512, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 120274028, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 736512, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:28.708 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 736512, 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 120274028, 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:28.710 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 68009, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 587708777, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 68009, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 587708777, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:28.712 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 68009, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 68009, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 68009, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 68009, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.713 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:28.714 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 10846, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1184694, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 10846, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 445, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1184694, 2024-08-16T19:26:28.715 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55e3378c0960": { 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:28.716 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 86004, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 583473382, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 86004, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:28.717 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 43759, 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 583473382, 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.718 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:28.719 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-16T19:26:29.644 INFO:tasks.check_counter:Daemon mds.b mds.exported=26 2024-08-16T19:26:29.645 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2024-08-16T19:26:29.645 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=457 2024-08-16T19:26:29.645 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=356 2024-08-16T19:26:29.645 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-16T19:26:29.645 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-08-16T19:26:29.646 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-08-16T19:26:29.646 DEBUG:tasks.check_counter:Getting stats from e 2024-08-16T19:26:29.646 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-16T19:26:29.904 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:26:30.284 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:30 smithi155 ceph-mon[37413]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:30.299 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:30 smithi062 ceph-mon[32993]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:30.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:30 smithi161 ceph-mon[37576]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.7 MiB/s wr, 282 op/s 2024-08-16T19:26:30.393 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-16T19:26:30.393 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:30.393 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 685, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 620, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 14515498, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 283116, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 2, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.084897204, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.025491250, 2024-08-16T19:26:30.394 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.065019286, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.007231288, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 620, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.030967419, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049947 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.395 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1859584, 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 192480 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1031, 2024-08-16T19:26:30.396 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1261, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 42217017, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 973501, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.118207125, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.029533376, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.053782396, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.025671920, 2024-08-16T19:26:30.397 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1258, 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.044769459, 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000035587 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.398 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 646592 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2371, 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2373, 2024-08-16T19:26:30.399 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 38363906, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 754036, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.213661487, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.063141910, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.069454336, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.064036800, 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:30.400 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2371, 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.086444213, 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000036458 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.401 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-16T19:26:30.402 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6268, 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.147462489, 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000023526 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:30.403 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-08-16T19:26:30.404 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:30.405 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 7960, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 62, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 7898, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-08-16T19:26:30.406 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 7872, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 14, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-08-16T19:26:30.407 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-08-16T19:26:30.408 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:30.409 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-08-16T19:26:30.410 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-08-16T19:26:30.411 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-08-16T19:26:30.412 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:30.413 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-08-16T19:26:30.414 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-08-16T19:26:30.415 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 41927, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-08-16T19:26:30.416 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 56, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 5, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 96393280, 2024-08-16T19:26:30.417 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 96382159, 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 41927 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.418 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 946, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 7835, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 6889, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 67, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 69, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 2, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 7960, 2024-08-16T19:26:30.419 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 8003, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 43, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 61452, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215328 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.420 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.421 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:30.422 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.423 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.424 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-16T19:26:30.425 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.426 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.427 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-16T19:26:30.428 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.429 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.430 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.431 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.432 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:30.433 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.434 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.435 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-16T19:26:30.436 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.437 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.438 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.439 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-08-16T19:26:30.440 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 2230, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:30.441 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:30.442 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:30.443 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:30.444 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:30.445 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 262609, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1114, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:30.446 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 196, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-16T19:26:30.447 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 8091652, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 56342, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-16T19:26:30.448 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 3684, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 3684, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3684, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.596945833, 2024-08-16T19:26:30.449 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000704925 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3684, 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3685 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 3684, 2024-08-16T19:26:30.450 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 3684, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 2069, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1613, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-08-16T19:26:30.451 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:30.452 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-16T19:26:30.453 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-16T19:26:30.454 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-16T19:26:30.455 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 113, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 25, 2024-08-16T19:26:30.456 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 9, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 9, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.457 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.458 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:30.459 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4083, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 94705839, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4083, 2024-08-16T19:26:30.460 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4083, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 94705839, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.461 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:30.462 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3684, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 92214575, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3684, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 92214575, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.463 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:30.464 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3684, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3684, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3684, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3684, 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:30.465 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:30.466 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-16T19:26:30.467 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55b3aa924b40": { 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:30.468 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:30.469 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:30.470 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.471 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:30.471 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-08-16T19:26:31.909 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:31.910 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:31.910 DEBUG:tasks.check_counter:Getting stats from h 2024-08-16T19:26:31.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-16T19:26:32.167 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:26:32.289 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:32 smithi155 ceph-mon[37413]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 291 op/s 2024-08-16T19:26:32.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:32 smithi161 ceph-mon[37576]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 291 op/s 2024-08-16T19:26:32.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:32 smithi062 ceph-mon[32993]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 291 op/s 2024-08-16T19:26:33.031 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2467, 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2469, 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 227728273, 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 784516, 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2024-08-16T19:26:33.032 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.281181490, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.066426116, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.142336247, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.055315612, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2467, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.072830518, 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000029521 2024-08-16T19:26:33.033 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:33.034 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1246, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1181, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 174049177, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 461094, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:33.035 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.199183214, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.034149054, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.156865494, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.018685208, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1181, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.043307524, 2024-08-16T19:26:33.036 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000036670 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1859472, 2024-08-16T19:26:33.037 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 192000 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1156, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1384, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 184851666, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 925197, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:33.038 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.198416041, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.030458522, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.130290583, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.027849785, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1381, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.047759018, 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000034582 2024-08-16T19:26:33.039 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 558224 2024-08-16T19:26:33.040 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7211, 2024-08-16T19:26:33.041 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.151045111, 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000020946 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:33.042 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-08-16T19:26:33.043 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:33.044 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 11453, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1057, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 10396, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 9943, 2024-08-16T19:26:33.045 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 12, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-08-16T19:26:33.046 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-08-16T19:26:33.047 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 729, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 219574819, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:33.048 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-08-16T19:26:33.049 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.050 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:33.051 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:33.052 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:33.053 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-08-16T19:26:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 86008, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 143, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 8, 2024-08-16T19:26:33.055 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 42521852, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 587721334, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 587695705, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 86008 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 470, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 12948, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 12478, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 64, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 173, 2024-08-16T19:26:33.057 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 109, 2024-08-16T19:26:33.058 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 11453, 2024-08-16T19:26:33.058 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 13598, 2024-08-16T19:26:33.058 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 2145, 2024-08-16T19:26:33.058 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-08-16T19:26:33.058 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 72072, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 215328 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-08-16T19:26:33.059 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.060 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:33.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:33.063 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.064 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.065 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.066 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-16T19:26:33.067 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-16T19:26:33.069 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-16T19:26:33.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.073 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.074 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-16T19:26:33.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 6, 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 4, 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 4, 2024-08-16T19:26:33.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3503, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:33.081 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:33.082 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:33.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:33.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:33.085 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 261678, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 660, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:33.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 196, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 9520424, 2024-08-16T19:26:33.087 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 59747, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-16T19:26:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 4467, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 26, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 4, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 4467, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4467, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 4.126913905, 2024-08-16T19:26:33.089 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000923867 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4467, 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 4468 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 4467, 2024-08-16T19:26:33.090 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 4467, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 2582, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1882, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-08-16T19:26:33.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:33.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:33.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-16T19:26:33.094 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-16T19:26:33.095 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 113, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 25, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 11, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 11, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-08-16T19:26:33.096 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:33.097 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:33.098 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 0, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:33.099 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 4865, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 586168203, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 4865, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:33.100 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4865, 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 586168203, 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:33.101 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 4467, 2024-08-16T19:26:33.102 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 583541908, 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4467, 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 583541908, 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.103 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:33.104 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 4467, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 4467, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 4467, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 4467, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.105 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:33.106 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-16T19:26:33.107 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x56499510eb40": { 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:33.111 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-16T19:26:33.995 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:33.996 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:33.996 DEBUG:tasks.check_counter:Getting stats from k 2024-08-16T19:26:33.997 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-16T19:26:34.256 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.b/config 2024-08-16T19:26:34.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:34 smithi155 ceph-mon[37413]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 495 KiB/s wr, 99 op/s 2024-08-16T19:26:34.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:34 smithi161 ceph-mon[37576]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 495 KiB/s wr, 99 op/s 2024-08-16T19:26:34.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:34 smithi062 ceph-mon[32993]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 495 KiB/s wr, 99 op/s 2024-08-16T19:26:34.759 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-08-16T19:26:34.759 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:34.759 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 82549, 2024-08-16T19:26:34.760 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 114503, 2024-08-16T19:26:34.760 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 26445296, 2024-08-16T19:26:34.760 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 102591082, 2024-08-16T19:26:34.760 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 16, 2024-08-16T19:26:34.760 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 4.512945143, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 1.946417926, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 2.834758858, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.166493655, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 114498, 2024-08-16T19:26:34.761 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.514747333, 2024-08-16T19:26:34.762 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000030697 2024-08-16T19:26:34.762 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.762 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:34.762 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 170884, 2024-08-16T19:26:34.762 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.024795422, 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000145 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 4112, 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 436848 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:34.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 35724, 2024-08-16T19:26:34.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 32528, 2024-08-16T19:26:34.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 9471070, 2024-08-16T19:26:34.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 56483851, 2024-08-16T19:26:34.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-08-16T19:26:34.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:34.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 1.922883313, 2024-08-16T19:26:34.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.737762176, 2024-08-16T19:26:34.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 1.012646606, 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.292222555, 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 32527, 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.676724294, 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000051548 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:34.767 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 43836, 2024-08-16T19:26:34.768 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.007128227, 2024-08-16T19:26:34.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000162 2024-08-16T19:26:34.768 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.768 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-08-16T19:26:34.768 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 127648 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 185435, 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 124468, 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 58557623, 2024-08-16T19:26:34.769 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 45355226, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 11, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 9.025700632, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 3.964124117, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 6.506853099, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.191771924, 2024-08-16T19:26:34.770 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 124469, 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2.685559548, 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000021576 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 297829, 2024-08-16T19:26:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.040500836, 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000135 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1862544, 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 220768 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-08-16T19:26:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 19041, 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 13.198394882, 2024-08-16T19:26:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000693156 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-08-16T19:26:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 11058, 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.789795822, 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000071423 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 53390, 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 43258, 2024-08-16T19:26:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 43258, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 113.298751854, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002619139 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 10132, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 3, 2024-08-16T19:26:34.776 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 11, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 724, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 83, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 625, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 16, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 605, 2024-08-16T19:26:34.777 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 446, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 124, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 125, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 15, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 99933, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 48253, 2024-08-16T19:26:34.778 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 10127, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 176, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 1662, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 30097, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 7, 2024-08-16T19:26:34.779 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 349, 2024-08-16T19:26:34.780 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 8, 2024-08-16T19:26:34.780 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 649, 2024-08-16T19:26:34.780 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:34.780 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:34.780 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 729, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 219574819, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:34.781 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:34.782 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:34.782 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:34.782 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:34.782 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:34.782 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 5, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 6667, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 28691, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 17316, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 4982, 2024-08-16T19:26:34.783 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 3241, 2024-08-16T19:26:34.784 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 37879 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 691, 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 1294, 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 703, 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 1288, 2024-08-16T19:26:34.786 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 734, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 5407, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 5407, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-08-16T19:26:34.787 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_path": 14, 2024-08-16T19:26:34.788 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_quiesce_inode": 735, 2024-08-16T19:26:34.789 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:34.789 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 8, 2024-08-16T19:26:34.789 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-08-16T19:26:34.789 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "evlrg": 0, 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 37897, 2024-08-16T19:26:34.790 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 37897, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 51, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-08-16T19:26:34.791 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 51, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "segmjr": 5, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 84269195, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-08-16T19:26:34.792 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 9587, 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 29.886770821, 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003117426 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.793 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 559, 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 6351, 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 5792, 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 139, 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 688, 2024-08-16T19:26:34.794 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 549, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 724, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 11436, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 10712, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 125, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 6533, 2024-08-16T19:26:34.795 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 6408, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 74872, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 223520 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 82123, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 45, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 53390, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 987, 2024-08-16T19:26:34.796 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 9800, 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5389, 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 44.200910505, 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008202061 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-08-16T19:26:34.797 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 293, 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.390329268, 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001332181 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.798 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-08-16T19:26:34.799 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 12528, 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 28.425785114, 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002268980 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.800 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000157883, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000157883 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.801 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.802 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.803 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-08-16T19:26:34.804 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.805 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001128097, 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000564048 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 17730, 2024-08-16T19:26:34.806 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 24.049905807, 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001356452 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1839, 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 5.028688593, 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002734469 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.807 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:34.808 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.809 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:34.810 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 25, 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.084039480, 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003361579 2024-08-16T19:26:34.811 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:34.812 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 44, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.072261898, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001642315 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.813 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-08-16T19:26:34.814 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.814 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.814 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.814 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.814 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5407, 2024-08-16T19:26:34.815 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 11.045545209, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002042823 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.816 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 4, 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-08-16T19:26:34.817 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 9075, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 4537, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 939, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-08-16T19:26:34.818 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:34.819 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:34.820 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:34.821 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-08-16T19:26:34.822 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 335297, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 646, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-08-16T19:26:34.823 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 196, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:34.824 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 2278512, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 20733, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:34.825 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 22262, 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 81357494, 2024-08-16T19:26:34.826 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 33, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 22262, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 22262, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 81.696296962, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003669764 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-08-16T19:26:34.827 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 22262, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 23033 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 22262, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 488, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 21774, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-08-16T19:26:34.828 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 10, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 485, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 10072, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 314, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-08-16T19:26:34.829 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-08-16T19:26:34.830 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 10845, 2024-08-16T19:26:34.830 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-08-16T19:26:34.830 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:34.830 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:34.830 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 20, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-08-16T19:26:34.831 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-08-16T19:26:34.832 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-08-16T19:26:34.833 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 113, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 25, 2024-08-16T19:26:34.834 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 991, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 6, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 290 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.835 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 639 2024-08-16T19:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 10, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 5, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed_ops": 10814, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 5407, 2024-08-16T19:26:34.837 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 303704, 2024-08-16T19:26:34.838 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 67114826, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 303704, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 303704, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 67114826, 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:34.839 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:34.840 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 22262, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 81948225, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 22262, 2024-08-16T19:26:34.841 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 81948225, 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:34.842 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 22262, 2024-08-16T19:26:34.843 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 22262, 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 22262, 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 22262, 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.844 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 5407, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 590779, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 5407, 2024-08-16T19:26:34.845 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 485, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 590779, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.846 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55d603b74960": { 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 37897, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 80074891, 2024-08-16T19:26:34.847 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 37897, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 9587, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 80074891, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-08-16T19:26:34.848 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-08-16T19:26:34.849 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-08-16T19:26:34.849 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.849 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-08-16T19:26:34.849 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds.exported=7 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds.imported=8 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=691 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=1294 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:35.544 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:35.545 DEBUG:tasks.check_counter:Getting stats from c 2024-08-16T19:26:35.545 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-16T19:26:35.813 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:26:36.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:36 smithi161 ceph-mon[37576]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 508 KiB/s wr, 106 op/s 2024-08-16T19:26:36.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:36 smithi155 ceph-mon[37413]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 508 KiB/s wr, 106 op/s 2024-08-16T19:26:36.335 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-16T19:26:36.335 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:36.335 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 187345, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 133184, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 56647108, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 73017754, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 8.746371140, 2024-08-16T19:26:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 3.421926243, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 6.208702219, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.423585430, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 133182, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.894901453, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000029244 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:36.337 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 293089, 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.042775829, 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000145 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.338 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 85930, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 124021, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 28196129, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 128935190, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 14, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 10, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 5.101496042, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 2.192366788, 2024-08-16T19:26:36.339 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 2.972599758, 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.317708208, 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 124018, 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5.229755563, 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000042169 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.340 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 171078, 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.025720746, 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000150 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1864240, 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 198496 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:36.341 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 49506, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 43971, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 14862765, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 22056383, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 14, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 3.047744690, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.397274461, 2024-08-16T19:26:36.342 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.910583215, 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.103496135, 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 43971, 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.263426401, 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000074217 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:36.343 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 82986, 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.014561014, 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000175 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 4272, 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 561376 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-16T19:26:36.344 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 25387, 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 14.811584516, 2024-08-16T19:26:36.345 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000583431 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 13607, 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.973408917, 2024-08-16T19:26:36.346 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000071537 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 53415, 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 42874, 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42874, 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 158.656595381, 2024-08-16T19:26:36.347 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003700531 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 10540, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 20, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 1, 2024-08-16T19:26:36.348 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1211, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 193, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 987, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 31, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 931, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 163, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 149, 2024-08-16T19:26:36.349 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 149, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 21, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 108496, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 52041, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 10539, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 178, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:36.350 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 1949, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 29629, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 1, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 44, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 14, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 835, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:36.351 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 729, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 219574819, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:36.352 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 7, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 4860, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 29502, 2024-08-16T19:26:36.353 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 14471, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 5956, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 2142, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 40607 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.354 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 662, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 1352, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 730, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 1350, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 861, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:36.355 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 6852, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 6852, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:36.356 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 14, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 838, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 5, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 1, 2024-08-16T19:26:36.357 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 41934, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-16T19:26:36.358 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 41934, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 56, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-16T19:26:36.359 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 56, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 5, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 96433363, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-16T19:26:36.360 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 12690, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 35.202485989, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002774033 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 1038, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 8009, 2024-08-16T19:26:36.361 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 6971, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 152, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 716, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 564, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1211, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 9742, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 8531, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 149, 2024-08-16T19:26:36.362 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 7315, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 7166, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 74536, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 91629, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 1040, 2024-08-16T19:26:36.363 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 53415, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 1551, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 15820, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6842, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 38.041483533, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005559994 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.364 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 799, 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 11.301198534, 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.014144178 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.366 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 10375, 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 54.867095272, 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005288394 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:36.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.370 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-16T19:26:36.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000552401, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000552401 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15165, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 16.240675376, 2024-08-16T19:26:36.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001070931 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2047, 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 24.953059248, 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.012190063 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.374 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:36.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5, 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.025379728, 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005075945 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 788, 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.302162066, 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000383454 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-16T19:26:36.379 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.380 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6852, 2024-08-16T19:26:36.381 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 12.924989223, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001886308 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-08-16T19:26:36.383 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 9122, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 9122, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 893, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:36.384 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:36.385 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:36.387 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:36.388 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 426894, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 1062, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-16T19:26:36.389 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 196, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:36.390 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 3795902, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 34454, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-16T19:26:36.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 28207, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 93922318, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 28207, 2024-08-16T19:26:36.392 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 28207, 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 101.657765052, 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003603990 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 28207, 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 28980 2024-08-16T19:26:36.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 28207, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 458, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 27749, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-08-16T19:26:36.394 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 458, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 13148, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 295, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 1, 2024-08-16T19:26:36.395 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 13737, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:36.396 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-16T19:26:36.397 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-16T19:26:36.398 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 113, 2024-08-16T19:26:36.399 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 25, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1022, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-08-16T19:26:36.400 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 289 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.401 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 729 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 5, 2024-08-16T19:26:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 13704, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 6852, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 322777, 2024-08-16T19:26:36.403 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 70631346, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 322777, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 322777, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 70631346, 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:36.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:36.405 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 28207, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 94670858, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 28207, 2024-08-16T19:26:36.406 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 94670858, 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:36.407 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 28207, 2024-08-16T19:26:36.408 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 28207, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 28207, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 28207, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.409 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 6852, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 748540, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:36.410 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 6852, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 458, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 748540, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55a2980c4960": { 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 574, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 41934, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 92239059, 2024-08-16T19:26:36.412 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 41934, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 12690, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 92238485, 2024-08-16T19:26:36.413 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:36.414 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-16T19:26:36.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:36 smithi062 ceph-mon[32993]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 508 KiB/s wr, 106 op/s 2024-08-16T19:26:37.128 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-08-16T19:26:37.128 INFO:tasks.check_counter:Daemon mds.c mds.imported=14 2024-08-16T19:26:37.128 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=662 2024-08-16T19:26:37.128 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=1352 2024-08-16T19:26:37.129 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-08-16T19:26:37.129 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:37.129 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:37.129 DEBUG:tasks.check_counter:Getting stats from f 2024-08-16T19:26:37.129 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-16T19:26:37.388 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:26:37.914 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2233, 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2235, 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 45475184, 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 710101, 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-08-16T19:26:37.915 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.204356299, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.056291974, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.070188267, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.061811615, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2233, 2024-08-16T19:26:37.916 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.076260223, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000034151 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:37.917 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 785, 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1013, 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 46338434, 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 808424, 2024-08-16T19:26:37.918 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.109949859, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.024973143, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.053168246, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.020915449, 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:37.919 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1011, 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.043618514, 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000043143 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:37.920 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 560640 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1059, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 996, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 48162852, 2024-08-16T19:26:37.921 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 403510, 2024-08-16T19:26:37.922 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-08-16T19:26:37.922 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-08-16T19:26:38.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.128219971, 2024-08-16T19:26:38.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.033168520, 2024-08-16T19:26:38.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.091094650, 2024-08-16T19:26:38.003 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.015573998, 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 995, 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.035843068, 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000036023 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.004 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1857104, 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 193312 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-16T19:26:38.005 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6189, 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.146147468, 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000023614 2024-08-16T19:26:38.006 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.007 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.008 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-16T19:26:38.009 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 12270, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 30, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 12240, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 12228, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-08-16T19:26:38.010 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 10, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-08-16T19:26:38.011 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:38.012 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:38.013 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:38.014 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-08-16T19:26:38.015 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-08-16T19:26:38.016 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-08-16T19:26:38.017 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.018 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 57318, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-08-16T19:26:38.019 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 75, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 6, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 141287393, 2024-08-16T19:26:38.020 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 141286819, 2024-08-16T19:26:38.021 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-16T19:26:38.021 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.021 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.021 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.021 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 57318 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 615, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 12189, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 11574, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 48, 2024-08-16T19:26:38.022 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 49, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 1, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 12270, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 12367, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 97, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-08-16T19:26:38.023 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 63476, 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-08-16T19:26:38.024 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.025 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.026 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-16T19:26:38.027 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.028 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.029 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:38.029 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.029 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.029 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.029 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.030 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-16T19:26:38.031 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.031 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.031 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.031 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.031 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:38.032 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.032 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.032 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.032 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.032 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-16T19:26:38.033 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.034 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.034 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.034 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-16T19:26:38.035 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.036 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.037 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.038 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.039 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.040 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.041 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.042 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-16T19:26:38.043 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.046 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-08-16T19:26:38.047 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 4807, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-16T19:26:38.048 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:38.049 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:38.050 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:38.051 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:38.051 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:38.051 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:38.054 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-16T19:26:38.055 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 257672, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 747, 2024-08-16T19:26:38.056 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:38.057 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 196, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 10301330, 2024-08-16T19:26:38.058 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 65051, 2024-08-16T19:26:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-16T19:26:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:38.059 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 3673, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-08-16T19:26:38.060 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 3673, 2024-08-16T19:26:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-16T19:26:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3673, 2024-08-16T19:26:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.980934384, 2024-08-16T19:26:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000811580 2024-08-16T19:26:38.061 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3673, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3674 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 3673, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 3673, 2024-08-16T19:26:38.062 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-08-16T19:26:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-16T19:26:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-16T19:26:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 2076, 2024-08-16T19:26:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-08-16T19:26:38.063 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1595, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-08-16T19:26:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:38.065 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:38.066 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-16T19:26:38.067 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-16T19:26:38.068 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 113, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 25, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 10, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 11, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 1, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-16T19:26:38.069 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-08-16T19:26:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-08-16T19:26:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-08-16T19:26:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-16T19:26:38.070 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-08-16T19:26:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-16T19:26:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-16T19:26:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.071 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-16T19:26:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-08-16T19:26:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-16T19:26:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:38.072 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:38.073 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 4073, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 139586727, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:38.074 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 4073, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4073, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 139586727, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.075 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:38.076 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3673, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 137094592, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3673, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 137094592, 2024-08-16T19:26:38.077 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:38.078 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 3673, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 3673, 2024-08-16T19:26:38.079 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3673, 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 3673, 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:38.080 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:38.081 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.082 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55a414ac6b40": { 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:38.083 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:38.084 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.085 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:38.085 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-16T19:26:38.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:38 smithi161 ceph-mon[37576]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:38.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:38 smithi155 ceph-mon[37413]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:38.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:38 smithi062 ceph-mon[32993]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:38.803 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:38.804 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:38.804 DEBUG:tasks.check_counter:Getting stats from i 2024-08-16T19:26:38.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-16T19:26:39.064 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:26:39.747 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-16T19:26:39.747 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 214986, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 177918, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 63090933, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 81812050, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 10.803118028, 2024-08-16T19:26:39.748 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 4.518260267, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 7.394071057, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.383505543, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 177917, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5.600996755, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000031480 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.749 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 367357, 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.053800414, 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000146 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 437904 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.750 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 118375, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 177386, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 36633708, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 178189786, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 20, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 12, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 6.750279998, 2024-08-16T19:26:39.751 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 2.896548033, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 3.881986912, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.439776740, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 177379, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 7.212207095, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000040659 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:39.752 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 240766, 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.036291805, 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000150 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3744, 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 130608 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:39.753 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 38309, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 44656, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 12554320, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 56895549, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 2.956763757, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.233840222, 2024-08-16T19:26:39.754 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.282742726, 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.546589751, 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 44659, 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.583156690, 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000080233 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:39.755 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 48612, 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009299516, 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000191 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1867792, 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 197888 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.756 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-16T19:26:39.757 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-16T19:26:39.757 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-16T19:26:39.757 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.757 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-16T19:26:39.757 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 34331, 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 21.929183530, 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000638757 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:39.758 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 22330, 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.734291387, 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000077666 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 77659, 2024-08-16T19:26:39.759 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 68753, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 68753, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 405.322704011, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005895345 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 8905, 2024-08-16T19:26:39.760 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 851, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 50, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 682, 2024-08-16T19:26:39.761 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 119, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 741, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 206, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 141, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 141, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 17, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 150442, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 63674, 2024-08-16T19:26:39.762 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 8904, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 179, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 977, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 29216, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 4, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 122, 2024-08-16T19:26:39.763 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 6, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 442, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 729, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 219574819, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-08-16T19:26:39.764 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:39.765 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 15, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 5273, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 41927, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 16181, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 6483, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 1991, 2024-08-16T19:26:39.766 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 60738 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 547, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 1884, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 1011, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 1810, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 755, 2024-08-16T19:26:39.767 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 11505, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 11505, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:39.768 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 14, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 908, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 6, 2024-08-16T19:26:39.769 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 57322, 2024-08-16T19:26:39.770 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 57322, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 75, 2024-08-16T19:26:39.771 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 75, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 6, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-16T19:26:39.772 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 141293217, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 17374, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 49.898830463, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002872040 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.773 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 699, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 12327, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 11628, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 136, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 955, 2024-08-16T19:26:39.774 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 819, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 853, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 19213, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 18360, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 139, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 12399, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 12260, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 72320, 2024-08-16T19:26:39.775 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 135437, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 1109, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 77659, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 1147, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 21080, 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-16T19:26:39.776 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11541, 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 218.286240362, 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.018913979 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1338, 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 12.018452603, 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008982401 2024-08-16T19:26:39.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-16T19:26:39.779 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 18005, 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 41.486475939, 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002304164 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.780 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.781 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.782 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.783 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-16T19:26:39.784 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.785 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-16T19:26:39.786 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000851841, 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000425920 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 24988, 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 28.321331724, 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001133397 2024-08-16T19:26:39.787 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1246, 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 34.621908875, 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.027786443 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.788 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-16T19:26:39.789 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.790 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.790 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.790 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.790 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.791 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 28, 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.245548022, 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008769572 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.792 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 100, 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.145190636, 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001451906 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.793 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.794 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11505, 2024-08-16T19:26:39.795 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 70.196704009, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006101408 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.796 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-08-16T19:26:39.797 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 13241, 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 13241, 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 893, 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:39.798 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:39.799 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:39.800 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:39.801 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:39.802 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 306702, 2024-08-16T19:26:39.803 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 590, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 264, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 3, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:39.804 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 196, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:39.805 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 2710313, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 22962, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.806 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 42625, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 139453360, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 32, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 42625, 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-16T19:26:39.807 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42625, 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 144.277824778, 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003384817 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42625, 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 43430 2024-08-16T19:26:39.808 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 42625, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 643, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 41982, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-08-16T19:26:39.809 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 643, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 18017, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 371, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 23054, 2024-08-16T19:26:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:39.811 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-16T19:26:39.812 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-16T19:26:39.813 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 113, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-16T19:26:39.814 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 25, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 965, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 350 2024-08-16T19:26:39.815 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-16T19:26:39.816 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 816 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 5, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 23010, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 11505, 2024-08-16T19:26:39.817 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 371666, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 78803892, 2024-08-16T19:26:39.818 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 371666, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 371666, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 78803892, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.819 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:39.820 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 42625, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 140708381, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 42625, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 140708381, 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:39.821 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-16T19:26:39.822 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 42625, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 42625, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 42625, 2024-08-16T19:26:39.823 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 42625, 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:39.824 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 11505, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1255021, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:39.825 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 11505, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 643, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1255021, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.826 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x558a0eee0960": { 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 5824, 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 57322, 2024-08-16T19:26:39.827 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 137098913, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 57322, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 17374, 2024-08-16T19:26:39.828 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 137093089, 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:39.829 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-16T19:26:40.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:40 smithi155 ceph-mon[37413]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:40.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:40 smithi161 ceph-mon[37576]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:40.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:40 smithi062 ceph-mon[32993]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 16 op/s 2024-08-16T19:26:40.769 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-08-16T19:26:40.769 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-08-16T19:26:40.769 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=547 2024-08-16T19:26:40.769 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1884 2024-08-16T19:26:40.770 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-16T19:26:40.770 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:40.770 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:40.770 DEBUG:tasks.check_counter:Getting stats from l 2024-08-16T19:26:40.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-16T19:26:41.029 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.c/config 2024-08-16T19:26:41.523 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-16T19:26:41.523 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-08-16T19:26:41.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 185090, 2024-08-16T19:26:41.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 117835, 2024-08-16T19:26:41.523 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 57200554, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 52820259, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 8.490180279, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 3.395244870, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 6.412185760, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.191357091, 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:41.524 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 117833, 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.498721021, 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000038178 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 287771, 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.041295837, 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000143 2024-08-16T19:26:41.525 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 13896, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 14200, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 5496536, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 8719194, 2024-08-16T19:26:41.526 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.875449605, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.324804064, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.551792332, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.089571559, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 14199, 2024-08-16T19:26:41.527 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.811556746, 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000127583 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 19588, 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004446200, 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000226 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.528 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1850736, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 641232 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 65966, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 84326, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 20132797, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 86146784, 2024-08-16T19:26:41.529 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 17, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 9, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 4.363944993, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.925868303, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 2.606752793, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.186902532, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 84324, 2024-08-16T19:26:41.530 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5.428618314, 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000064378 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 126446, 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.024485068, 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000193 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.531 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 6320, 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 135632 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-08-16T19:26:41.532 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 13137, 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 9.989839939, 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000760435 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-08-16T19:26:41.533 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7981, 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.608754299, 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000076275 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.534 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 30122, 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 29151, 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 29151, 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 100.697389090, 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003454337 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 970, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 1020, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 119, 2024-08-16T19:26:41.536 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 853, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 48, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 849, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 141, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 139, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 139, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 16, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 58402, 2024-08-16T19:26:41.537 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 32075, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 967, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 117, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 831, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 33731, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 4, 2024-08-16T19:26:41.538 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 106, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 9, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 788, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 729, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 219574819, 2024-08-16T19:26:41.539 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-08-16T19:26:41.540 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 7, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 4301, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 19177, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 11621, 2024-08-16T19:26:41.541 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 3693, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 2242, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 26271 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 550, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 1468, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 788, 2024-08-16T19:26:41.542 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 1380, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 600, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 3720, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 3720, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-08-16T19:26:41.543 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 14, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 672, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-08-16T19:26:41.544 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 8, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-08-16T19:26:41.545 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 25911, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 25911, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 35, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-08-16T19:26:41.546 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 35, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 4, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 59527842, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-08-16T19:26:41.547 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6589, 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 24.024903606, 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003646213 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 813, 2024-08-16T19:26:41.548 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 4651, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 3838, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 147, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 754, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 607, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 1020, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 7553, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 6533, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 139, 2024-08-16T19:26:41.549 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 4462, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 4323, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 67612, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 49325, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 5, 2024-08-16T19:26:41.550 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 30122, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 335, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 5801, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3697, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 50.045834842, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.013536877 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.551 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 254, 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.076103741, 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008173636 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.552 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-08-16T19:26:41.553 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6994, 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 17.555212901, 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002510039 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.554 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.555 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.556 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.557 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-08-16T19:26:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001499491, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000499830 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 13461, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 18.752871606, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001393126 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 985, 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.415868205, 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003467886 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.561 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-08-16T19:26:41.562 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.563 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.564 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-08-16T19:26:41.565 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37, 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.111670801, 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003018129 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.566 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.567 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3720, 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 8.738327503, 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002349012 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.568 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.569 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 8433, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 4216, 2024-08-16T19:26:41.570 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 937, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-08-16T19:26:41.571 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-16T19:26:41.572 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-16T19:26:41.573 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-08-16T19:26:41.574 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 351655, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 865, 2024-08-16T19:26:41.575 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-08-16T19:26:41.576 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 9704, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 196, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 3119134, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 27909, 2024-08-16T19:26:41.577 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-08-16T19:26:41.578 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 15757, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 56224201, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 15757, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15757, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 66.460540956, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004217842 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15757, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 16466 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 15757, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 440, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 15317, 2024-08-16T19:26:41.580 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 440, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 7029, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 297, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-08-16T19:26:41.581 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 7465, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-08-16T19:26:41.582 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-08-16T19:26:41.583 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-08-16T19:26:41.584 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-08-16T19:26:41.585 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 113, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 25, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-08-16T19:26:41.586 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 948, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 257 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-08-16T19:26:41.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 573 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2024-08-16T19:26:41.588 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 5, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 7440, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 3720, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:41.589 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 264948, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 58958943, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 264948, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:41.590 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 264948, 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 58958943, 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.591 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:41.592 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 15757, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 56630705, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 15757, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 56630705, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.593 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-08-16T19:26:41.594 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 15757, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 15757, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 15757, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 15757, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.595 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 3720, 2024-08-16T19:26:41.596 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 406504, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 3720, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 440, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 406504, 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:41.597 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55c5b0e3a960": { 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 3862, 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-08-16T19:26:41.598 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 25911, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 55333538, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 25911, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 6589, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 55329676, 2024-08-16T19:26:41.599 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-16T19:26:41.600 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-16T19:26:42.289 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:42 smithi161 ceph-mon[37576]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-08-16T19:26:42.290 INFO:tasks.check_counter:Daemon mds.l mds.exported=4 2024-08-16T19:26:42.290 INFO:tasks.check_counter:Daemon mds.l mds.imported=9 2024-08-16T19:26:42.290 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=550 2024-08-16T19:26:42.291 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=1468 2024-08-16T19:26:42.291 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-08-16T19:26:42.291 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-08-16T19:26:42.291 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-08-16T19:26:42.291 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-16T19:26:42.300 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-16T19:26:42.301 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:26:42.301 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-16T19:26:42.324 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-08-16T19:26:42.324 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-16T19:26:42.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:42 smithi155 ceph-mon[37413]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-08-16T19:26:42.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-16T19:26:42.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:42 smithi062 ceph-mon[32993]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-08-16T19:26:42.707 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 -- 172.21.15.62:0/1907330274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 msgr2=0x7f9b84102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/1907330274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b84102f00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009900 tx=0x7f9b7402a410 comp rx=0 tx=0).stop 2024-08-16T19:26:43.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 -- 172.21.15.62:0/1907330274 shutdown_connections 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/1907330274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 0x7f9b8410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/1907330274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 0x7f9b8410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/1907330274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b84102f00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 -- 172.21.15.62:0/1907330274 >> 172.21.15.62:0/1907330274 conn(0x7f9b840fc500 msgr2=0x7f9b840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 -- 172.21.15.62:0/1907330274 shutdown_connections 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 -- 172.21.15.62:0/1907330274 wait complete. 2024-08-16T19:26:43.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.179+0000 7f9b8a17b640 1 Processor -- start 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 -- start start 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 0x7f9b84198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 0x7f9b8419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84111e20 con 0x7f9b84102b20 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84111ca0 con 0x7f9b8410c020 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b8a17b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84111fa0 con 0x7f9b841034d0 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50158/0 (socket says 172.21.15.62:50158) 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 -- 172.21.15.62:0/2042890557 learned_addr learned my addr 172.21.15.62:0/2042890557 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b83fff640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 0x7f9b8419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b82ffd640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 0x7f9b84198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 -- 172.21.15.62:0/2042890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 msgr2=0x7f9b84198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 0x7f9b84198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 -- 172.21.15.62:0/2042890557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 msgr2=0x7f9b8419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 0x7f9b8419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.180+0000 7f9b837fe640 1 -- 172.21.15.62:0/2042890557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8419d290 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b837fe640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006010 tx=0x7f9b740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7403f030 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b74003ce0 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8419d520 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74035070 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.181+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a4e60 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.182+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8406a4c0 con 0x7f9b84102b20 2024-08-16T19:26:43.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.183+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9b74042030 con 0x7f9b84102b20 2024-08-16T19:26:43.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.184+0000 7f9b80ff9640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 0x7f9b6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:43.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.185+0000 7f9b82ffd640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 0x7f9b6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:43.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.185+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f9b740efff0 con 0x7f9b84102b20 2024-08-16T19:26:43.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.185+0000 7f9b82ffd640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 0x7f9b6c0a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b8410d460 tx=0x7f9b7800b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:43.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.188+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9b740bc290 con 0x7f9b84102b20 2024-08-16T19:26:43.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.381+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9b84102f00 con 0x7f9b84102b20 2024-08-16T19:26:43.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.382+0000 7f9b80ff9640 1 -- 172.21.15.62:0/2042890557 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v113) ==== 81+0+939 (secure 0 0 0) 0x7f9b84102f00 con 0x7f9b84102b20 2024-08-16T19:26:43.383 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/151448061 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.383 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/727495888 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.383 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3894293084 2024-08-17T19:01:33.379605+0000 2024-08-16T19:26:43.383 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/2033319205 2024-08-17T19:01:57.573528+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/795341867 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3941316438 2024-08-17T19:01:33.379605+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/4100589739 2024-08-17T19:01:33.379605+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1123206672 2024-08-17T19:01:57.573528+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/727495888 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.384 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:26:43.385 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/3183618050 2024-08-17T19:01:57.573528+0000 2024-08-16T19:26:43.385 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6800/572676055 2024-08-17T19:01:33.379605+0000 2024-08-16T19:26:43.385 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/1715350407 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.385 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6801/3198450894 2024-08-17T19:01:57.573528+0000 2024-08-16T19:26:43.385 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:0/439686452 2024-08-17T19:10:28.302138+0000 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:listed 16 entries 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 msgr2=0x7f9b6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 0x7f9b6c0a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9b8410d460 tx=0x7f9b7800b390 comp rx=0 tx=0).stop 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 msgr2=0x7f9b841982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006010 tx=0x7f9b740042f0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 shutdown_connections 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f9b6c0a2cf0 0x7f9b6c0a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b8410c020 0x7f9b8419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b841034d0 0x7f9b84198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 --2- 172.21.15.62:0/2042890557 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b84102b20 0x7f9b841982e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 >> 172.21.15.62:0/2042890557 conn(0x7f9b840fc500 msgr2=0x7f9b840fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 shutdown_connections 2024-08-16T19:26:43.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:43.384+0000 7f9b8a17b640 1 -- 172.21.15.62:0/2042890557 wait complete. 2024-08-16T19:26:44.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:44 smithi062 ceph-mon[32993]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-08-16T19:26:44.113 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:44 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/2042890557' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:26:44.141 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-16T19:26:44.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:26:44.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-16T19:26:44.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:44 smithi155 ceph-mon[37413]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-08-16T19:26:44.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:44 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/2042890557' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:26:44.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:44 smithi161 ceph-mon[37576]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-08-16T19:26:44.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:44 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/2042890557' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-16T19:26:44.395 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:26:44.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:26:44.395 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-16T19:26:44.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:26:44.419 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-08-16T19:26:44.496 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-16T19:26:44.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:26:44.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-16T19:26:44.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-16T19:26:44.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-08-16T19:26:44.685 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-16T19:26:44.695 INFO:tasks.cephadm:Teardown begin 2024-08-16T19:26:44.696 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-16T19:26:44.696 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:26:44.962 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 -- 172.21.15.62:0/3092672612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 msgr2=0x7fa958100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 --2- 172.21.15.62:0/3092672612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958100f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa948009900 tx=0x7fa94802a480 comp rx=0 tx=0).stop 2024-08-16T19:26:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 -- 172.21.15.62:0/3092672612 shutdown_connections 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 --2- 172.21.15.62:0/3092672612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa958102c20 0x7fa95810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 --2- 172.21.15.62:0/3092672612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa9581022a0 0x7fa9581026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 --2- 172.21.15.62:0/3092672612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958100f30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 -- 172.21.15.62:0/3092672612 >> 172.21.15.62:0/3092672612 conn(0x7fa9580fc500 msgr2=0x7fa9580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 -- 172.21.15.62:0/3092672612 shutdown_connections 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.442+0000 7fa95e566640 1 -- 172.21.15.62:0/3092672612 wait complete. 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 Processor -- start 2024-08-16T19:26:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 -- start start 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 0x7fa958197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 0x7fa95819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110d00 con 0x7fa958100b50 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110b80 con 0x7fa9581022a0 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa95e566640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa958110e80 con 0x7fa958102c20 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa957fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa957fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50172/0 (socket says 172.21.15.62:50172) 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.443+0000 7fa957fff640 1 -- 172.21.15.62:0/4081041536 learned_addr learned my addr 172.21.15.62:0/4081041536 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa95cb03640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 0x7fa95819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa94ffff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 0x7fa958197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 -- 172.21.15.62:0/4081041536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 msgr2=0x7fa95819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 0x7fa95819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 -- 172.21.15.62:0/4081041536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 msgr2=0x7fa958197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 0x7fa958197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 -- 172.21.15.62:0/4081041536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95819bfb0 con 0x7fa958100b50 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa95cb03640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 0x7fa95819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa94ffff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 0x7fa958197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa957fff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa948006010 tx=0x7fa948003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94803f070 con 0x7fa958100b50 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948003e30 con 0x7fa958100b50 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95819c240 con 0x7fa958100b50 2024-08-16T19:26:45.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948035030 con 0x7fa958100b50 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.444+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581a3b80 con 0x7fa958100b50 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.445+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958101ee0 con 0x7fa958100b50 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.447+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa948042070 con 0x7fa958100b50 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.447+0000 7fa955ffb640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 0x7fa9440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.447+0000 7fa94ffff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 0x7fa9440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.447+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fa9480f0890 con 0x7fa958100b50 2024-08-16T19:26:45.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.448+0000 7fa94ffff640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 0x7fa9440a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa958106f30 tx=0x7fa940005d90 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:45.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.451+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa9480bd0c0 con 0x7fa958100b50 2024-08-16T19:26:45.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.666+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa958100f30 con 0x7fa958100b50 2024-08-16T19:26:45.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.666+0000 7fa955ffb640 1 -- 172.21.15.62:0/4081041536 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+12010 (secure 0 0 0) 0x7fa9480bc9c0 con 0x7fa958100b50 2024-08-16T19:26:45.668 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:45.668 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"btime":"2024-08-16T19:26:05:936217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:26:05.891993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:26:45.668 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 75 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 msgr2=0x7fa9440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 0x7fa9440a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa958106f30 tx=0x7fa940005d90 comp rx=0 tx=0).stop 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 msgr2=0x7fa958197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa948006010 tx=0x7fa948003c10 comp rx=0 tx=0).stop 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 shutdown_connections 2024-08-16T19:26:45.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fa9440a2cf0 0x7fa9440a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa958102c20 0x7fa95819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581022a0 0x7fa958197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 --2- 172.21.15.62:0/4081041536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa958100b50 0x7fa958197000 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 >> 172.21.15.62:0/4081041536 conn(0x7fa9580fc500 msgr2=0x7fa9580fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 shutdown_connections 2024-08-16T19:26:45.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:45.669+0000 7fa95e566640 1 -- 172.21.15.62:0/4081041536 wait complete. 2024-08-16T19:26:46.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:46 smithi155 ceph-mon[37413]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-08-16T19:26:46.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:46 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/4081041536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:46.330 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:46 smithi062 ceph-mon[32993]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-08-16T19:26:46.331 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:46 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/4081041536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:46.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-16T19:26:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:46 smithi161 ceph-mon[37576]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 73 KiB/s wr, 22 op/s 2024-08-16T19:26:46.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:46 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/4081041536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:46.629 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:47.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 -- 172.21.15.62:0/896443592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 msgr2=0x7fe65010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 --2- 172.21.15.62:0/896443592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe65010e820 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe648009cd0 tx=0x7fe64802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 -- 172.21.15.62:0/896443592 shutdown_connections 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 --2- 172.21.15.62:0/896443592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe65010e820 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 --2- 172.21.15.62:0/896443592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6501009d0 0x7fe650106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 --2- 172.21.15.62:0/896443592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe650100020 0x7fe650100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 -- 172.21.15.62:0/896443592 >> 172.21.15.62:0/896443592 conn(0x7fe650076f40 msgr2=0x7fe6500fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 -- 172.21.15.62:0/896443592 shutdown_connections 2024-08-16T19:26:47.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.085+0000 7fe656699640 1 -- 172.21.15.62:0/896443592 wait complete. 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 Processor -- start 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 -- start start 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe650100020 0x7fe650100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6501009d0 0x7fe6501014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:47.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111d20 con 0x7fe650106c80 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111ba0 con 0x7fe6501009d0 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.086+0000 7fe656699640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111ea0 con 0x7fe650100020 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50200/0 (socket says 172.21.15.62:50200) 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 -- 172.21.15.62:0/1303049468 learned_addr learned my addr 172.21.15.62:0/1303049468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe64ffff640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe650100020 0x7fe650100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe64f7fe640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6501009d0 0x7fe6501014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 -- 172.21.15.62:0/1303049468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe650100020 msgr2=0x7fe650100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe650100020 0x7fe650100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 -- 172.21.15.62:0/1303049468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6501009d0 msgr2=0x7fe6501014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6501009d0 0x7fe6501014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 -- 172.21.15.62:0/1303049468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6501a65e0 con 0x7fe650106c80 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe654c36640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe648033040 tx=0x7fe648003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648004040 con 0x7fe650106c80 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6480046c0 con 0x7fe650106c80 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648030eb0 con 0x7fe650106c80 2024-08-16T19:26:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6501a6870 con 0x7fe650106c80 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.087+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501a6db0 con 0x7fe650106c80 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.089+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650100400 con 0x7fe650106c80 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.090+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe6480041e0 con 0x7fe650106c80 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.091+0000 7fe64d7fa640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 0x7fe6380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.091+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fe64802e030 con 0x7fe650106c80 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.091+0000 7fe64ffff640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 0x7fe6380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:47.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.091+0000 7fe64ffff640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 0x7fe6380a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe64000abe0 tx=0x7fe64000a2d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:47.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.095+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6480bd680 con 0x7fe650106c80 2024-08-16T19:26:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.329+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe65002d5e0 con 0x7fe650106c80 2024-08-16T19:26:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.330+0000 7fe64d7fa640 1 -- 172.21.15.62:0/1303049468 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+12010 (secure 0 0 0) 0x7fe6480bcf80 con 0x7fe650106c80 2024-08-16T19:26:47.331 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:47.332 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"btime":"2024-08-16T19:26:05:936217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1364784278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1364784278},{"type":"v1","addr":"172.21.15.62:6835","nonce":1364784278}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/3300090633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":3300090633},{"type":"v1","addr":"172.21.15.62:6837","nonce":3300090633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-16T19:08:05.356964+0000","modified":"2024-08-16T19:26:05.891993+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24476,"mds_1":24469,"mds_2":24451,"mds_3":24407,"mds_4":24400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6839/659126409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":659126409},{"type":"v1","addr":"172.21.15.62:6839","nonce":659126409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6841/3131957026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6840","nonce":3131957026},{"type":"v1","addr":"172.21.15.62:6841","nonce":3131957026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.161:6833/1822368312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":1822368312},{"type":"v1","addr":"172.21.15.161:6833","nonce":1822368312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.155:6833/2391473540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2391473540},{"type":"v1","addr":"172.21.15.155:6833","nonce":2391473540}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/3063588307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":3063588307},{"type":"v1","addr":"172.21.15.155:6835","nonce":3063588307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3346321531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3346321531},{"type":"v1","addr":"172.21.15.161:6835","nonce":3346321531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3410199866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3410199866},{"type":"v1","addr":"172.21.15.155:6837","nonce":3410199866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.161:6837/3769663917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3769663917},{"type":"v1","addr":"172.21.15.161:6837","nonce":3769663917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.161:6839/2650096635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":2650096635},{"type":"v1","addr":"172.21.15.161:6839","nonce":2650096635}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6839/2045382457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":2045382457},{"type":"v1","addr":"172.21.15.155:6839","nonce":2045382457}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24469,24451,24407,24400]},"id":1}]} 2024-08-16T19:26:47.332 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 75 2024-08-16T19:26:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.332+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 msgr2=0x7fe6380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.332+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 0x7fe6380a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe64000abe0 tx=0x7fe64000a2d0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.332+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 msgr2=0x7fe650101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.332+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe648033040 tx=0x7fe648003970 comp rx=0 tx=0).stop 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 shutdown_connections 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe6380a2cf0 0x7fe6380a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe650106c80 0x7fe650101a20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6501009d0 0x7fe6501014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 --2- 172.21.15.62:0/1303049468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe650100020 0x7fe650100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:47.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 >> 172.21.15.62:0/1303049468 conn(0x7fe650076f40 msgr2=0x7fe6501053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 shutdown_connections 2024-08-16T19:26:47.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:47.333+0000 7fe656699640 1 -- 172.21.15.62:0/1303049468 wait complete. 2024-08-16T19:26:48.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-16T19:26:48.320 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1303049468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:26:48.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:48 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:26:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1303049468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:26:48.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:26:48.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:48 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:26:48.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:48.442 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1303049468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-16T19:26:48.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-16T19:26:48.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-16T19:26:48.443 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:48 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' 2024-08-16T19:26:48.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 -- 172.21.15.62:0/2801367095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0101150 msgr2=0x7f7fa010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:48.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/2801367095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0101150 0x7f7fa010d680 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009cd0 tx=0x7f7f9802a8e0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 -- 172.21.15.62:0/2801367095 shutdown_connections 2024-08-16T19:26:48.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/2801367095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa0101150 0x7f7fa010d680 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/2801367095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa01007b0 0x7f7fa0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/2801367095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 0x7f7fa0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 -- 172.21.15.62:0/2801367095 >> 172.21.15.62:0/2801367095 conn(0x7f7fa00fc4d0 msgr2=0x7f7fa00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 -- 172.21.15.62:0/2801367095 shutdown_connections 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 -- 172.21.15.62:0/2801367095 wait complete. 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.818+0000 7f7fa65c5640 1 Processor -- start 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 -- start start 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa0101150 0x7f7fa0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 0x7f7fa01a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0110bb0 con 0x7f7fa01007b0 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0110a30 con 0x7f7fa0101150 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa65c5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0110d30 con 0x7f7fa01067b0 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50218/0 (socket says 172.21.15.62:50218) 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 -- 172.21.15.62:0/1821036153 learned_addr learned my addr 172.21.15.62:0/1821036153 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f97fff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa0101150 0x7f7fa0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7fa4b62640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 0x7f7fa01a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 -- 172.21.15.62:0/1821036153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 msgr2=0x7f7fa01a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 0x7f7fa01a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 -- 172.21.15.62:0/1821036153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa0101150 msgr2=0x7f7fa0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa0101150 0x7f7fa0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 -- 172.21.15.62:0/1821036153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a4580 con 0x7f7fa01007b0 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9ffff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000e9f0 tx=0x7f7f9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:48.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.819+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000cd80 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.820+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f90005500 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.820+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a4750 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.820+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90010690 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.820+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a4c10 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.821+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0100c10 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.822+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f900046c0 con 0x7f7fa01007b0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.822+0000 7f7f9dffb640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 0x7f7f8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.823+0000 7f7f97fff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 0x7f7f8c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:48.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.823+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f7f900cfd40 con 0x7f7fa01007b0 2024-08-16T19:26:48.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.823+0000 7f7f97fff640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 0x7f7f8c0a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7fa006b010 tx=0x7f7f8800b590 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:48.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:48.826+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7f9009c570 con 0x7f7fa01007b0 2024-08-16T19:26:49.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.004+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fa0108d20 con 0x7f7fa01007b0 2024-08-16T19:26:49.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.006+0000 7f7f9dffb640 1 -- 172.21.15.62:0/1821036153 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+25389 (secure 0 0 0) 0x7f7f9009be70 con 0x7f7fa01007b0 2024-08-16T19:26:49.007 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-08-16T19:26:49.008 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":113,"fsid":"bd001ebc-5c01-11ef-bccf-c7b262605968","created":"2024-08-16T19:01:02.785646+0000","modified":"2024-08-16T19:26:11.635281+0000","last_up_change":"2024-08-16T19:07:14.326853+0000","last_in_change":"2024-08-16T19:06:55.318066+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-16T19:03:58.075413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-16T19:08:01.384527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-16T19:08:03.560289+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-16T19:08:10.039515+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c0cec841-7b80-4f4a-909c-74544c63b869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6803","nonce":3542478147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6805","nonce":3542478147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6809","nonce":3542478147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":3542478147},{"type":"v1","addr":"172.21.15.62:6807","nonce":3542478147}]},"public_addr":"172.21.15.62:6803/3542478147","cluster_addr":"172.21.15.62:6805/3542478147","heartbeat_back_addr":"172.21.15.62:6809/3542478147","heartbeat_front_addr":"172.21.15.62:6807/3542478147","state":["exists","up"]},{"osd":1,"uuid":"54e23d69-41c1-420c-aa46-2833c651059e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6801","nonce":471904740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6803","nonce":471904740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6807","nonce":471904740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":471904740},{"type":"v1","addr":"172.21.15.155:6805","nonce":471904740}]},"public_addr":"172.21.15.155:6801/471904740","cluster_addr":"172.21.15.155:6803/471904740","heartbeat_back_addr":"172.21.15.155:6807/471904740","heartbeat_front_addr":"172.21.15.155:6805/471904740","state":["exists","up"]},{"osd":2,"uuid":"42c0d89e-6fb6-4a52-94d1-8198a745128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6801","nonce":3525036557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6803","nonce":3525036557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6807","nonce":3525036557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3525036557},{"type":"v1","addr":"172.21.15.161:6805","nonce":3525036557}]},"public_addr":"172.21.15.161:6801/3525036557","cluster_addr":"172.21.15.161:6803/3525036557","heartbeat_back_addr":"172.21.15.161:6807/3525036557","heartbeat_front_addr":"172.21.15.161:6805/3525036557","state":["exists","up"]},{"osd":3,"uuid":"5d4b830f-2036-4a2e-ab45-59ed8a735275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6811","nonce":1745722345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6813","nonce":1745722345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6817","nonce":1745722345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":1745722345},{"type":"v1","addr":"172.21.15.62:6815","nonce":1745722345}]},"public_addr":"172.21.15.62:6811/1745722345","cluster_addr":"172.21.15.62:6813/1745722345","heartbeat_back_addr":"172.21.15.62:6817/1745722345","heartbeat_front_addr":"172.21.15.62:6815/1745722345","state":["exists","up"]},{"osd":4,"uuid":"047fb0e0-b0ac-45b7-8cd9-23e62d1f52de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6809","nonce":34440659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6811","nonce":34440659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6815","nonce":34440659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":34440659},{"type":"v1","addr":"172.21.15.155:6813","nonce":34440659}]},"public_addr":"172.21.15.155:6809/34440659","cluster_addr":"172.21.15.155:6811/34440659","heartbeat_back_addr":"172.21.15.155:6815/34440659","heartbeat_front_addr":"172.21.15.155:6813/34440659","state":["exists","up"]},{"osd":5,"uuid":"5fdaca8b-0017-49e5-9731-7c7c13c5b463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6809","nonce":2837279313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6811","nonce":2837279313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6815","nonce":2837279313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":2837279313},{"type":"v1","addr":"172.21.15.161:6813","nonce":2837279313}]},"public_addr":"172.21.15.161:6809/2837279313","cluster_addr":"172.21.15.161:6811/2837279313","heartbeat_back_addr":"172.21.15.161:6815/2837279313","heartbeat_front_addr":"172.21.15.161:6813/2837279313","state":["exists","up"]},{"osd":6,"uuid":"a966574b-ec4f-4f19-848c-a755ecbda258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6819","nonce":3718256322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6821","nonce":3718256322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6825","nonce":3718256322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3718256322},{"type":"v1","addr":"172.21.15.62:6823","nonce":3718256322}]},"public_addr":"172.21.15.62:6819/3718256322","cluster_addr":"172.21.15.62:6821/3718256322","heartbeat_back_addr":"172.21.15.62:6825/3718256322","heartbeat_front_addr":"172.21.15.62:6823/3718256322","state":["exists","up"]},{"osd":7,"uuid":"61e8cc89-9d50-46d6-8d08-9762b73cdf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6817","nonce":785303840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6819","nonce":785303840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6823","nonce":785303840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":785303840},{"type":"v1","addr":"172.21.15.155:6821","nonce":785303840}]},"public_addr":"172.21.15.155:6817/785303840","cluster_addr":"172.21.15.155:6819/785303840","heartbeat_back_addr":"172.21.15.155:6823/785303840","heartbeat_front_addr":"172.21.15.155:6821/785303840","state":["exists","up"]},{"osd":8,"uuid":"31f59120-2671-4158-bf29-e39406478022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6817","nonce":2842601651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6819","nonce":2842601651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6823","nonce":2842601651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2842601651},{"type":"v1","addr":"172.21.15.161:6821","nonce":2842601651}]},"public_addr":"172.21.15.161:6817/2842601651","cluster_addr":"172.21.15.161:6819/2842601651","heartbeat_back_addr":"172.21.15.161:6823/2842601651","heartbeat_front_addr":"172.21.15.161:6821/2842601651","state":["exists","up"]},{"osd":9,"uuid":"b3ae18d6-96bf-4b37-9fa6-99129e4edf74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6827","nonce":851969557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6829","nonce":851969557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6833","nonce":851969557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":851969557},{"type":"v1","addr":"172.21.15.62:6831","nonce":851969557}]},"public_addr":"172.21.15.62:6827/851969557","cluster_addr":"172.21.15.62:6829/851969557","heartbeat_back_addr":"172.21.15.62:6833/851969557","heartbeat_front_addr":"172.21.15.62:6831/851969557","state":["exists","up"]},{"osd":10,"uuid":"b50d6aad-7475-4b81-911e-a3d08459d2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6825","nonce":1300242753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6827","nonce":1300242753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6831","nonce":1300242753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1300242753},{"type":"v1","addr":"172.21.15.155:6829","nonce":1300242753}]},"public_addr":"172.21.15.155:6825/1300242753","cluster_addr":"172.21.15.155:6827/1300242753","heartbeat_back_addr":"172.21.15.155:6831/1300242753","heartbeat_front_addr":"172.21.15.155:6829/1300242753","state":["exists","up"]},{"osd":11,"uuid":"25c71c88-59d5-4d2e-98dd-85072ddb3c0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6825","nonce":1433195011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6827","nonce":1433195011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6831","nonce":1433195011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1433195011},{"type":"v1","addr":"172.21.15.161:6829","nonce":1433195011}]},"public_addr":"172.21.15.161:6825/1433195011","cluster_addr":"172.21.15.161:6827/1433195011","heartbeat_back_addr":"172.21.15.161:6831/1433195011","heartbeat_front_addr":"172.21.15.161:6829/1433195011","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:13.336206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:34.011279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:03:54.586001+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:15.837801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:37.394304+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:04:58.421698+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:20.363272+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:05:42.169402+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:04.322441+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:27.321514+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:06:49.978683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-16T19:07:12.366921+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:0/151448061":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3894293084":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/2033319205":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/795341867":"2024-08-17T19:10:28.302138+0000","172.21.15.62:0/3941316438":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/4100589739":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1123206672":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6801/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:6800/727495888":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6800/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/3183618050":"2024-08-17T19:01:57.573528+0000","172.21.15.62:6800/572676055":"2024-08-17T19:01:33.379605+0000","172.21.15.62:0/1715350407":"2024-08-17T19:10:28.302138+0000","172.21.15.62:6801/3198450894":"2024-08-17T19:01:57.573528+0000","172.21.15.62:0/439686452":"2024-08-17T19:10:28.302138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 msgr2=0x7f7f8c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 0x7f7f8c0a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7fa006b010 tx=0x7f7f8800b590 comp rx=0 tx=0).stop 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 msgr2=0x7f7fa0198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000e9f0 tx=0x7f7f9000edf0 comp rx=0 tx=0).stop 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 shutdown_connections 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f7f8c0a2cf0 0x7f7f8c0a51b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa01067b0 0x7f7fa01a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fa0101150 0x7f7fa0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 --2- 172.21.15.62:0/1821036153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fa01007b0 0x7f7fa0198fc0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 >> 172.21.15.62:0/1821036153 conn(0x7f7fa00fc4d0 msgr2=0x7f7fa00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 shutdown_connections 2024-08-16T19:26:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:49.008+0000 7f7fa65c5640 1 -- 172.21.15.62:0/1821036153 wait complete. 2024-08-16T19:26:49.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:49 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1821036153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:49.333 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:49 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1821036153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:49.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:49 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1821036153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-16T19:26:49.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-16T19:26:49.923 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:50.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mon[37413]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:50.346 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-mon[32993]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:50.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-mon[37576]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 -- 172.21.15.62:0/2050417497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34101067b0 msgr2=0x7f3410106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 --2- 172.21.15.62:0/2050417497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34101067b0 0x7f3410106b90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3400009930 tx=0x7f340002a420 comp rx=0 tx=0).stop 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 -- 172.21.15.62:0/2050417497 shutdown_connections 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 --2- 172.21.15.62:0/2050417497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3410101150 0x7f341010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 --2- 172.21.15.62:0/2050417497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 0x7f3410100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 --2- 172.21.15.62:0/2050417497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34101067b0 0x7f3410106b90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 -- 172.21.15.62:0/2050417497 >> 172.21.15.62:0/2050417497 conn(0x7f34100fc4d0 msgr2=0x7f34100fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 -- 172.21.15.62:0/2050417497 shutdown_connections 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.429+0000 7f3414bc3640 1 -- 172.21.15.62:0/2050417497 wait complete. 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 Processor -- start 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 -- start start 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 0x7f3410102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34101067b0 0x7f3410102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3410110bb0 con 0x7f3410101150 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f3410110a30 con 0x7f34101007b0 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f3414bc3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3410110d30 con 0x7f34101067b0 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f340dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f340dd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50224/0 (socket says 172.21.15.62:50224) 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f340dd9b640 1 -- 172.21.15.62:0/1882364052 learned_addr learned my addr 172.21.15.62:0/1882364052 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f340e59c640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 0x7f3410102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.430+0000 7f340ed9d640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34101067b0 0x7f3410102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 -- 172.21.15.62:0/1882364052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34101067b0 msgr2=0x7f3410102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34101067b0 0x7f3410102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 -- 172.21.15.62:0/1882364052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 msgr2=0x7f3410102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 0x7f3410102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 -- 172.21.15.62:0/1882364052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34101a6620 con 0x7f3410101150 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f340dd9b640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d950 tx=0x7f33f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017070 con 0x7f3410101150 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f8009c30 con 0x7f3410101150 2024-08-16T19:26:50.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34101a6910 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f800f6b0 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.431+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34101a6e50 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.432+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3410100c10 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.433+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f33f80109d0 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.434+0000 7f34077fe640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 0x7f33fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.434+0000 7f340e59c640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 0x7f33fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.434+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f33f80cf4d0 con 0x7f3410101150 2024-08-16T19:26:50.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.435+0000 7f340e59c640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 0x7f33fc0a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3400009900 tx=0x7f34000058d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.438+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f33f8009de0 con 0x7f3410101150 2024-08-16T19:26:50.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.653+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34101a7020 con 0x7f3410101150 2024-08-16T19:26:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.656+0000 7f34077fe640 1 -- 172.21.15.62:0/1882364052 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f33f809bd00 con 0x7f3410101150 2024-08-16T19:26:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-16T19:26:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.659+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 msgr2=0x7f33fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.659+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 0x7f33fc0a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3400009900 tx=0x7f34000058d0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.659+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 msgr2=0x7f34101028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.659+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d950 tx=0x7f33f800de10 comp rx=0 tx=0).stop 2024-08-16T19:26:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 shutdown_connections 2024-08-16T19:26:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33fc0a2cf0 0x7f33fc0a51b0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3400009900 tx=0x7f34000058d0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34101067b0 0x7f3410102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3410101150 0x7f34101028c0 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d950 tx=0x7f33f800de10 comp rx=0 tx=0).stop 2024-08-16T19:26:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 --2- 172.21.15.62:0/1882364052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34101007b0 0x7f3410102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:50.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 >> 172.21.15.62:0/1882364052 conn(0x7f34100fc4d0 msgr2=0x7f34101045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:50.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 shutdown_connections 2024-08-16T19:26:50.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:50.660+0000 7f3414bc3640 1 -- 172.21.15.62:0/1882364052 wait complete. 2024-08-16T19:26:50.980 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.980 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.980 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mds[67110]: -- 172.21.15.155:0/1591102028 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56382609b860 con 0x563826079000 2024-08-16T19:26:50.980 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: starting mds.e at 2024-08-16T19:26:50.981 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.981 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.981 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mds[67412]: -- 172.21.15.155:0/1588482137 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ec9a1d8680 con 0x55ec9a1ad000 2024-08-16T19:26:50.981 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mds[67412]: -- 172.21.15.155:0/1588482137 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec9a1cf860 con 0x55ec9a1ad000 2024-08-16T19:26:50.981 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: starting mds.h at 2024-08-16T19:26:50.982 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.982 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.982 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mds[67714]: -- 172.21.15.155:0/1731436927 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563de8613c20 con 0x563de85f0800 2024-08-16T19:26:50.982 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: starting mds.b at 2024-08-16T19:26:50.982 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.983 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.983 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-mds[66790]: -- 172.21.15.155:0/4161003387 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e41fbf860 con 0x557e41f9d000 2024-08-16T19:26:50.983 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:26:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: starting mds.k at 2024-08-16T19:26:50.997 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a[32989]: 2024-08-16T19:26:50.654+0000 7f210ccc9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-16T19:26:50.997 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.997 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.998 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-mds[73683]: -- 172.21.15.62:0/1870548973 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df6815dc20 con 0x55df6813b000 2024-08-16T19:26:50.998 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: starting mds.a at 2024-08-16T19:26:50.998 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:50.998 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:50.999 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-mds[73284]: -- 172.21.15.62:0/1173018153 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5c014d860 con 0x55b5c012b400 2024-08-16T19:26:50.999 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:26:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: starting mds.g at 2024-08-16T19:26:51.047 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:51.047 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:51.047 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: starting mds.i at 2024-08-16T19:26:51.047 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:51.048 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:51.048 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-mds[67384]: -- 172.21.15.161:0/1597219483 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a25ab1860 con 0x555a25a8f400 2024-08-16T19:26:51.048 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: starting mds.c at 2024-08-16T19:26:51.048 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:51.048 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:51.049 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-mds[67084]: -- 172.21.15.161:0/2764790526 <== mon.0 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56044fa55860 con 0x56044fa33400 2024-08-16T19:26:51.049 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: starting mds.f at 2024-08-16T19:26:51.049 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: ignoring --setuser ceph since I am not root 2024-08-16T19:26:51.049 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: ignoring --setgroup ceph since I am not root 2024-08-16T19:26:51.050 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-mds[66766]: -- 172.21.15.161:0/232755207 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580f73fd860 con 0x5580f73db000 2024-08-16T19:26:51.050 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:26:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: starting mds.l at 2024-08-16T19:26:51.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-16T19:26:51.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-16T19:26:51.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-16T19:26:51.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-16T19:26:51.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: osdmap e114: 12 total, 12 up, 12 in 2024-08-16T19:26:51.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:51 smithi155 ceph-mon[37413]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-16T19:26:51.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-16T19:26:51.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-16T19:26:51.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-16T19:26:51.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-16T19:26:51.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: osdmap e114: 12 total, 12 up, 12 in 2024-08-16T19:26:51.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:51 smithi161 ceph-mon[37576]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-16T19:26:51.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-16T19:26:51.496 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-16T19:26:51.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-16T19:26:51.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1882364052' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-16T19:26:51.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: osdmap e114: 12 total, 12 up, 12 in 2024-08-16T19:26:51.497 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:51 smithi062 ceph-mon[32993]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-16T19:26:51.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-16T19:26:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 22 op/s 2024-08-16T19:26:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6838/4014077057,v1:172.21.15.62:6840/4014077057] up:boot 2024-08-16T19:26:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.62:6839/1990668250,v1:172.21.15.62:6841/1990668250] up:boot 2024-08-16T19:26:52.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6833/897066780,v1:172.21.15.155:6835/897066780] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6832/1606016661,v1:172.21.15.155:6834/1606016661] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6832/3054154108,v1:172.21.15.161:6834/3054154108] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6833/465141903,v1:172.21.15.161:6835/465141903] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6836/2004009039,v1:172.21.15.161:6837/2004009039] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.161:6838/2722905111,v1:172.21.15.161:6839/2722905111] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6836/1090360588,v1:172.21.15.155:6837/1090360588] up:boot 2024-08-16T19:26:52.248 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: mds.? [v2:172.21.15.155:6838/1860447223,v1:172.21.15.155:6839/1860447223] up:boot 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:26:52.249 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:26:52.250 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:26:52.250 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:26:52.250 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:26:52.250 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:52 smithi062 ceph-mon[32993]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:26:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 22 op/s 2024-08-16T19:26:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6838/4014077057,v1:172.21.15.62:6840/4014077057] up:boot 2024-08-16T19:26:52.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.62:6839/1990668250,v1:172.21.15.62:6841/1990668250] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6833/897066780,v1:172.21.15.155:6835/897066780] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6832/1606016661,v1:172.21.15.155:6834/1606016661] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6832/3054154108,v1:172.21.15.161:6834/3054154108] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6833/465141903,v1:172.21.15.161:6835/465141903] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6836/2004009039,v1:172.21.15.161:6837/2004009039] up:boot 2024-08-16T19:26:52.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.161:6838/2722905111,v1:172.21.15.161:6839/2722905111] up:boot 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6836/1090360588,v1:172.21.15.155:6837/1090360588] up:boot 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: mds.? [v2:172.21.15.155:6838/1860447223,v1:172.21.15.155:6839/1860447223] up:boot 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:26:52.331 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:26:52.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:26:52.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:26:52.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:26:52.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:26:52.332 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:52 smithi155 ceph-mon[37413]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:26:52.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: pgmap v514: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 130 KiB/s wr, 22 op/s 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6838/4014077057,v1:172.21.15.62:6840/4014077057] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.62:6839/1990668250,v1:172.21.15.62:6841/1990668250] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6833/897066780,v1:172.21.15.155:6835/897066780] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6832/1606016661,v1:172.21.15.155:6834/1606016661] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6832/3054154108,v1:172.21.15.161:6834/3054154108] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6833/465141903,v1:172.21.15.161:6835/465141903] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6836/2004009039,v1:172.21.15.161:6837/2004009039] up:boot 2024-08-16T19:26:52.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.161:6838/2722905111,v1:172.21.15.161:6839/2722905111] up:boot 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6836/1090360588,v1:172.21.15.155:6837/1090360588] up:boot 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: mds.? [v2:172.21.15.155:6838/1860447223,v1:172.21.15.155:6839/1860447223] up:boot 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-16T19:26:52.390 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-16T19:26:52.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:52 smithi161 ceph-mon[37576]: from='mgr.15297 172.21.15.62:0/661370852' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-16T19:26:52.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.332+0000 7f0bb175b640 1 -- 172.21.15.62:0/805633469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 msgr2=0x7f0bac10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:52.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.332+0000 7f0bb175b640 1 --2- 172.21.15.62:0/805633469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac10e340 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009ca0 tx=0x7f0ba402a950 comp rx=0 tx=0).stop 2024-08-16T19:26:52.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- 172.21.15.62:0/805633469 shutdown_connections 2024-08-16T19:26:52.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- 172.21.15.62:0/805633469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac10e340 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- 172.21.15.62:0/805633469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 0x7f0bac10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- 172.21.15.62:0/805633469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 0x7f0bac102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- 172.21.15.62:0/805633469 >> 172.21.15.62:0/805633469 conn(0x7f0bac0fc4d0 msgr2=0x7f0bac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- 172.21.15.62:0/805633469 shutdown_connections 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- 172.21.15.62:0/805633469 wait complete. 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 Processor -- start 2024-08-16T19:26:52.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- start start 2024-08-16T19:26:52.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 0x7f0bac1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:52.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 0x7f0bac100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:52.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:52.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac111c80 con 0x7f0bac10bea0 2024-08-16T19:26:52.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac111b00 con 0x7f0bac102a70 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.333+0000 7f0bb175b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac111e00 con 0x7f0bac103390 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44506/0 (socket says 172.21.15.62:44506) 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 -- 172.21.15.62:0/401946985 learned_addr learned my addr 172.21.15.62:0/401946985 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0ba3fff640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 0x7f0bac100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:52.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0baaffd640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 0x7f0bac1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 -- 172.21.15.62:0/401946985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 msgr2=0x7f0bac100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 0x7f0bac100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 -- 172.21.15.62:0/401946985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 msgr2=0x7f0bac1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 0x7f0bac1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 -- 172.21.15.62:0/401946985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac1a6560 con 0x7f0bac10bea0 2024-08-16T19:26:52.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bab7fe640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0ba402ae70 tx=0x7f0ba402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4034040 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba4030920 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bac1a67f0 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40047b0 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.334+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bac1a6d20 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.335+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bac102e50 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.337+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0ba4037020 con 0x7f0bac10bea0 2024-08-16T19:26:52.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.337+0000 7f0ba8ff9640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 0x7f0b980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:52.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.337+0000 7f0baaffd640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 0x7f0b980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:52.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.337+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9867+0+0 (secure 0 0 0) 0x7f0ba40f0640 con 0x7f0bac10bea0 2024-08-16T19:26:52.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.338+0000 7f0baaffd640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 0x7f0b980a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00abe0 tx=0x7f0b9c00a2d0 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:52.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.341+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0ba40bced0 con 0x7f0bac10bea0 2024-08-16T19:26:52.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:52.553+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0bac1a6f80 con 0x7f0bac10bea0 2024-08-16T19:26:53.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.053+0000 7f0ba8ff9640 1 -- 172.21.15.62:0/401946985 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7f0ba40bc7d0 con 0x7f0bac10bea0 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 msgr2=0x7f0b980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 0x7f0b980a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00abe0 tx=0x7f0b9c00a2d0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 msgr2=0x7f0bac101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0ba402ae70 tx=0x7f0ba402f5b0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 shutdown_connections 2024-08-16T19:26:53.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f0b980a2d70 0x7f0b980a5230 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac10bea0 0x7f0bac101150 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bac103390 0x7f0bac100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 --2- 172.21.15.62:0/401946985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0bac102a70 0x7f0bac1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 >> 172.21.15.62:0/401946985 conn(0x7f0bac0fc4d0 msgr2=0x7f0bac0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 shutdown_connections 2024-08-16T19:26:53.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:53.055+0000 7f0bb175b640 1 -- 172.21.15.62:0/401946985 wait complete. 2024-08-16T19:26:53.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:53 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:26:53.414 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:53 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:26:53.414 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:53 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-16T19:26:53.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-16T19:26:54.013 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 18 op/s 2024-08-16T19:26:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-16T19:26:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-16T19:26:54.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: Cluster is now healthy 2024-08-16T19:26:54.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:26:54.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:54 smithi155 ceph-mon[37413]: fsmap 12 up:standby 2024-08-16T19:26:54.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 18 op/s 2024-08-16T19:26:54.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-16T19:26:54.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-16T19:26:54.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: Cluster is now healthy 2024-08-16T19:26:54.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:26:54.389 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:54 smithi161 ceph-mon[37576]: fsmap 12 up:standby 2024-08-16T19:26:54.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: pgmap v516: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 18 op/s 2024-08-16T19:26:54.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-16T19:26:54.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-16T19:26:54.391 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: Cluster is now healthy 2024-08-16T19:26:54.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/401946985' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-16T19:26:54.392 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:54 smithi062 ceph-mon[32993]: fsmap 12 up:standby 2024-08-16T19:26:54.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 -- 172.21.15.62:0/425812666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c4103390 msgr2=0x7f33c410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:54.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/425812666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c4103390 0x7f33c410b880 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009900 tx=0x7f33c002a480 comp rx=0 tx=0).stop 2024-08-16T19:26:54.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 -- 172.21.15.62:0/425812666 shutdown_connections 2024-08-16T19:26:54.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/425812666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c410bea0 0x7f33c410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:54.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/425812666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c4103390 0x7f33c410b880 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/425812666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4102a70 0x7f33c4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 -- 172.21.15.62:0/425812666 >> 172.21.15.62:0/425812666 conn(0x7f33c40fc4d0 msgr2=0x7f33c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 -- 172.21.15.62:0/425812666 shutdown_connections 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.476+0000 7f33cc2e1640 1 -- 172.21.15.62:0/425812666 wait complete. 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 Processor -- start 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 -- start start 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c4102a70 0x7f33c41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4103390 0x7f33c4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:54.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4111d00 con 0x7f33c410bea0 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4111b80 con 0x7f33c4102a70 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.477+0000 7f33cc2e1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4111e80 con 0x7f33c4103390 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44524/0 (socket says 172.21.15.62:44524) 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 -- 172.21.15.62:0/362859179 learned_addr learned my addr 172.21.15.62:0/362859179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:54.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33c987c640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4103390 0x7f33c4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca07d640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c4102a70 0x7f33c41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 -- 172.21.15.62:0/362859179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4103390 msgr2=0x7f33c4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4103390 0x7f33c4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 -- 172.21.15.62:0/362859179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c4102a70 msgr2=0x7f33c41981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c4102a70 0x7f33c41981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 -- 172.21.15.62:0/362859179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c419d180 con 0x7f33c410bea0 2024-08-16T19:26:54.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33ca87e640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b970 tx=0x7f33bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019070 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc00cb40 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c419d470 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc007650 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.478+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41a4d50 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.479+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c406ada0 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.481+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f33bc01c070 con 0x7f33c410bea0 2024-08-16T19:26:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.482+0000 7f33b77fe640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 0x7f33b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:54.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.482+0000 7f33ca07d640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 0x7f33b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:54.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.482+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9867+0+0 (secure 0 0 0) 0x7f33bc0cee50 con 0x7f33c410bea0 2024-08-16T19:26:54.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.482+0000 7f33ca07d640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 0x7f33b00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005fd0 tx=0x7f33b800b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:54.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.486+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f33bc09aff0 con 0x7f33c410bea0 2024-08-16T19:26:54.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:54.663+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f33c4106f10 con 0x7f33c410bea0 2024-08-16T19:26:55.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.055+0000 7f33b77fe640 1 -- 172.21.15.62:0/362859179 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f33bc09f2f0 con 0x7f33c410bea0 2024-08-16T19:26:55.056 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-08-16T19:26:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 msgr2=0x7f33b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 0x7f33b00a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005fd0 tx=0x7f33b800b390 comp rx=0 tx=0).stop 2024-08-16T19:26:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 msgr2=0x7f33c419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b970 tx=0x7f33bc00be30 comp rx=0 tx=0).stop 2024-08-16T19:26:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 shutdown_connections 2024-08-16T19:26:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7f33b00a2cf0 0x7f33b00a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c410bea0 0x7f33c419caa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33c4103390 0x7f33c4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 --2- 172.21.15.62:0/362859179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33c4102a70 0x7f33c41981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:55.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 >> 172.21.15.62:0/362859179 conn(0x7f33c40fc4d0 msgr2=0x7f33c40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:55.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 shutdown_connections 2024-08-16T19:26:55.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:55.058+0000 7f33cc2e1640 1 -- 172.21.15.62:0/362859179 wait complete. 2024-08-16T19:26:55.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:55 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:55.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:55 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:55.397 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:55 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:55.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-16T19:26:55.997 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:56 smithi155 ceph-mon[37413]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 103 KiB/s wr, 12 op/s 2024-08-16T19:26:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:56 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:56.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:56 smithi155 ceph-mon[37413]: osdmap e115: 12 total, 12 up, 12 in 2024-08-16T19:26:56.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:56 smithi062 ceph-mon[32993]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 103 KiB/s wr, 12 op/s 2024-08-16T19:26:56.375 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:56 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:56.376 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:56 smithi062 ceph-mon[32993]: osdmap e115: 12 total, 12 up, 12 in 2024-08-16T19:26:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:56 smithi161 ceph-mon[37576]: pgmap v517: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 103 KiB/s wr, 12 op/s 2024-08-16T19:26:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:56 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/362859179' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:56.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:56 smithi161 ceph-mon[37576]: osdmap e115: 12 total, 12 up, 12 in 2024-08-16T19:26:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 -- 172.21.15.62:0/1633345650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 msgr2=0x7fb6201026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 --2- 172.21.15.62:0/1633345650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb6201026e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb610009900 tx=0x7fb61002a480 comp rx=0 tx=0).stop 2024-08-16T19:26:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 -- 172.21.15.62:0/1633345650 shutdown_connections 2024-08-16T19:26:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 --2- 172.21.15.62:0/1633345650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620102c20 0x7fb62010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 --2- 172.21.15.62:0/1633345650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb6201026e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 --2- 172.21.15.62:0/1633345650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620100b50 0x7fb620100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 -- 172.21.15.62:0/1633345650 >> 172.21.15.62:0/1633345650 conn(0x7fb6200fc500 msgr2=0x7fb6200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 -- 172.21.15.62:0/1633345650 shutdown_connections 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.462+0000 7fb628669640 1 -- 172.21.15.62:0/1633345650 wait complete. 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 Processor -- start 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 -- start start 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620100b50 0x7fb6201982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:56.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 0x7fb62019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb620110cb0 con 0x7fb6201022a0 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fb620110b30 con 0x7fb620100b50 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb628669640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb620110e30 con 0x7fb620102c20 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44544/0 (socket says 172.21.15.62:44544) 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 -- 172.21.15.62:0/1316182702 learned_addr learned my addr 172.21.15.62:0/1316182702 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb626405640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620100b50 0x7fb6201982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:56.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb626c06640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 0x7fb62019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 -- 172.21.15.62:0/1316182702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 msgr2=0x7fb62019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 0x7fb62019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 -- 172.21.15.62:0/1316182702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620100b50 msgr2=0x7fb6201982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620100b50 0x7fb6201982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb625c04640 1 -- 172.21.15.62:0/1316182702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019d270 con 0x7fb6201022a0 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.463+0000 7fb626c06640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 0x7fb62019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb625c04640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb610003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61003f070 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610031c30 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019d500 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610035030 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.464+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a4e40 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.465+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620100f30 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.466+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb610042070 con 0x7fb6201022a0 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.467+0000 7fb6177fe640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 0x7fb60c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.467+0000 7fb626405640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 0x7fb60c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:56.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.467+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9342+0+0 (secure 0 0 0) 0x7fb6100f02f0 con 0x7fb6201022a0 2024-08-16T19:26:56.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.468+0000 7fb626405640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 0x7fb60c0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00b390 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:56.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.471+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb6100bcbe0 con 0x7fb6201022a0 2024-08-16T19:26:56.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:56.652+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb620109dd0 con 0x7fb6201022a0 2024-08-16T19:26:57.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.062+0000 7fb6177fe640 1 -- 172.21.15.62:0/1316182702 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7fb6100bc4e0 con 0x7fb6201022a0 2024-08-16T19:26:57.064 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-08-16T19:26:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.065+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 msgr2=0x7fb60c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.065+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 0x7fb60c0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c00b390 comp rx=0 tx=0).stop 2024-08-16T19:26:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.065+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 msgr2=0x7fb620198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:57.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.065+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb610003c10 comp rx=0 tx=0).stop 2024-08-16T19:26:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 shutdown_connections 2024-08-16T19:26:57.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fb60c0a2d70 0x7fb60c0a5230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb620102c20 0x7fb62019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6201022a0 0x7fb620198800 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 --2- 172.21.15.62:0/1316182702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb620100b50 0x7fb6201982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 >> 172.21.15.62:0/1316182702 conn(0x7fb6200fc500 msgr2=0x7fb620069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 shutdown_connections 2024-08-16T19:26:57.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:57.066+0000 7fb628669640 1 -- 172.21.15.62:0/1316182702 wait complete. 2024-08-16T19:26:57.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:57 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:57.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:57 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:57.454 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:57 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:57.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44 shell --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-16T19:26:58.142 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/mon.a/config 2024-08-16T19:26:58.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:58 smithi062 ceph-mon[32993]: pgmap v519: 97 pgs: 97 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:26:58.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:58 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:58.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:58 smithi062 ceph-mon[32993]: osdmap e116: 12 total, 12 up, 12 in 2024-08-16T19:26:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:58 smithi155 ceph-mon[37413]: pgmap v519: 97 pgs: 97 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:26:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:58 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:58.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:58 smithi155 ceph-mon[37413]: osdmap e116: 12 total, 12 up, 12 in 2024-08-16T19:26:58.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:58 smithi161 ceph-mon[37576]: pgmap v519: 97 pgs: 97 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:26:58.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:58 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/1316182702' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:26:58.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:58 smithi161 ceph-mon[37576]: osdmap e116: 12 total, 12 up, 12 in 2024-08-16T19:26:58.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 -- 172.21.15.62:0/1273928967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59010e200 msgr2=0x7fe590110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:58.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 --2- 172.21.15.62:0/1273928967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59010e200 0x7fe590110610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe580009ca0 tx=0x7fe58002a8c0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 -- 172.21.15.62:0/1273928967 shutdown_connections 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 --2- 172.21.15.62:0/1273928967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe59010e200 0x7fe590110610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 --2- 172.21.15.62:0/1273928967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe590101090 0x7fe59010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 --2- 172.21.15.62:0/1273928967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5901006e0 0x7fe590100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 -- 172.21.15.62:0/1273928967 >> 172.21.15.62:0/1273928967 conn(0x7fe5900fc500 msgr2=0x7fe5900fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 -- 172.21.15.62:0/1273928967 shutdown_connections 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.687+0000 7fe597d21640 1 -- 172.21.15.62:0/1273928967 wait complete. 2024-08-16T19:26:58.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 Processor -- start 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 -- start start 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901006e0 0x7fe59019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe59010e200 0x7fe59019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe590113fb0 con 0x7fe590101090 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 -- 0x7fe590113e30 con 0x7fe5901006e0 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe597d21640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe590114130 con 0x7fe59010e200 2024-08-16T19:26:58.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe5952bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe5952bc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44564/0 (socket says 172.21.15.62:44564) 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe5952bc640 1 -- 172.21.15.62:0/112550064 learned_addr learned my addr 172.21.15.62:0/112550064 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe595abd640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901006e0 0x7fe59019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.688+0000 7fe5962be640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe59010e200 0x7fe59019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 -- 172.21.15.62:0/112550064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe59010e200 msgr2=0x7fe59019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe59010e200 0x7fe59019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 -- 172.21.15.62:0/112550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901006e0 msgr2=0x7fe59019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:58.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901006e0 0x7fe59019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:58.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 -- 172.21.15.62:0/112550064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59019f440 con 0x7fe590101090 2024-08-16T19:26:58.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe5952bc640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe58400d950 tx=0x7fe58400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:58.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584017070 con 0x7fe590101090 2024-08-16T19:26:58.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59019f730 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58400de80 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400f620 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.689+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a7010 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.690+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59006a520 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.691+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe584009c30 con 0x7fe590101090 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.692+0000 7fe58effd640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 0x7fe5780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.692+0000 7fe595abd640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 0x7fe5780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-16T19:26:58.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.692+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8754+0+0 (secure 0 0 0) 0x7fe5840cf1f0 con 0x7fe590101090 2024-08-16T19:26:58.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.693+0000 7fe595abd640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 0x7fe5780a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe588009990 tx=0x7fe588009110 comp rx=0 tx=0).ready entity=mgr.15297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-16T19:26:58.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.696+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe58409bd20 con 0x7fe590101090 2024-08-16T19:26:58.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:58.875+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe590109390 con 0x7fe590101090 2024-08-16T19:26:59.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.071+0000 7fe58effd640 1 -- 172.21.15.62:0/112550064 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v117) ==== 178+0+0 (secure 0 0 0) 0x7fe58409b620 con 0x7fe590101090 2024-08-16T19:26:59.073 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data_ec' removed 2024-08-16T19:26:59.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 msgr2=0x7fe5780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 0x7fe5780a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe588009990 tx=0x7fe588009110 comp rx=0 tx=0).stop 2024-08-16T19:26:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 msgr2=0x7fe59019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-16T19:26:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe58400d950 tx=0x7fe58400dd50 comp rx=0 tx=0).stop 2024-08-16T19:26:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 shutdown_connections 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:6800/2270350447,v1:172.21.15.62:6801/2270350447] conn(0x7fe5780a2cf0 0x7fe5780a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe59010e200 0x7fe59019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe590101090 0x7fe59019a9d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 --2- 172.21.15.62:0/112550064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5901006e0 0x7fe59019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 >> 172.21.15.62:0/112550064 conn(0x7fe5900fc500 msgr2=0x7fe5900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.075+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 shutdown_connections 2024-08-16T19:26:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-08-16T19:26:59.076+0000 7fe597d21640 1 -- 172.21.15.62:0/112550064 wait complete. 2024-08-16T19:26:59.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:26:59 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:59.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:26:59 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:59.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:26:59 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-16T19:26:59.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:26:59.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:26:59.941 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:26:59.972 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-16T19:26:59.972 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-16T19:26:59.988 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-16T19:27:00.014 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-16T19:27:00.030 INFO:tasks.cephadm:Stopping all daemons... 2024-08-16T19:27:00.031 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-16T19:27:00.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a 2024-08-16T19:27:00.246 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 systemd[1]: Stopping Ceph mon.a for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:00.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-mon[32993]: pgmap v521: 33 pgs: 33 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:27:00.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-mon[32993]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:27:00.247 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-mon[32993]: osdmap e117: 12 total, 12 up, 12 in 2024-08-16T19:27:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:00 smithi155 ceph-mon[37413]: pgmap v521: 33 pgs: 33 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:27:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:00 smithi155 ceph-mon[37413]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:27:00.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:00 smithi155 ceph-mon[37413]: osdmap e117: 12 total, 12 up, 12 in 2024-08-16T19:27:00.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:00 smithi161 ceph-mon[37576]: pgmap v521: 33 pgs: 33 active+clean; 295 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-16T19:27:00.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:00 smithi161 ceph-mon[37576]: from='client.? 172.21.15.62:0/112550064' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-16T19:27:00.388 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:00 smithi161 ceph-mon[37576]: osdmap e117: 12 total, 12 up, 12 in 2024-08-16T19:27:00.503 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a[32989]: 2024-08-16T19:27:00.353+0000 7f21124d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:00.503 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a[32989]: 2024-08-16T19:27:00.353+0000 7f21124d4640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-16T19:27:00.503 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 podman[104289]: 2024-08-16 19:27:00.446012167 +0000 UTC m=+0.222148612 container died aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:00.829 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:27:00.501+0000 7fedb273b640 -1 client.15339: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-08-16T19:27:00.830 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: 2024-08-16T19:27:00.502+0000 7fedb273b640 -1 client.15339: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-08-16T19:27:00.830 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Exception in thread Thread-21: 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: Traceback (most recent call last): 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.run() 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.function(*self.args, **self.kwargs) 2024-08-16T19:27:00.831 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-08-16T19:27:00.832 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: self.prune_snapshots(sched) 2024-08-16T19:27:00.832 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x[33218]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-08-16T19:27:01.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 podman[104289]: 2024-08-16 19:27:00.829217032 +0000 UTC m=+0.605353522 container cleanup aef3f972923cdc2081f60b03cfc810ad3b59ae36208dc99ecf1c02c94ccaa4a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-16T19:27:01.139 INFO:journalctl@ceph.mon.a.smithi062.stdout:Aug 16 19:27:00 smithi062 bash[104289]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-a 2024-08-16T19:27:01.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.a.service' 2024-08-16T19:27:01.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:01.323 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-16T19:27:01.323 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-16T19:27:01.323 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.b 2024-08-16T19:27:01.661 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:01 smithi155 systemd[1]: Stopping Ceph mon.b for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:01.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-b[37409]: 2024-08-16T19:27:01.659+0000 7ffa7c858640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:01.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-b[37409]: 2024-08-16T19:27:01.659+0000 7ffa7c858640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-16T19:27:01.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:01 smithi155 podman[75467]: 2024-08-16 19:27:01.691269266 +0000 UTC m=+0.153291470 container died 8afd90ef91386c06f37056ea5bd0d532e93970a1f952fb2cd1b65da10616b706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-b, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-16T19:27:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:02 smithi155 podman[75467]: 2024-08-16 19:27:02.086256559 +0000 UTC m=+0.548278753 container cleanup 8afd90ef91386c06f37056ea5bd0d532e93970a1f952fb2cd1b65da10616b706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-b, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d863d21) 2024-08-16T19:27:02.329 INFO:journalctl@ceph.mon.b.smithi155.stdout:Aug 16 19:27:02 smithi155 bash[75467]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-b 2024-08-16T19:27:02.527 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.b.service' 2024-08-16T19:27:02.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:02.576 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-16T19:27:02.576 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-16T19:27:02.576 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.c 2024-08-16T19:27:02.888 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:02 smithi161 systemd[1]: Stopping Ceph mon.c for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:03.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-c[37572]: 2024-08-16T19:27:02.890+0000 7f847230f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:03.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-c[37572]: 2024-08-16T19:27:02.890+0000 7f847230f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-16T19:27:03.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:02 smithi161 podman[74736]: 2024-08-16 19:27:02.97236429 +0000 UTC m=+0.201466421 container died 171681b2b1a444c98748c56e6380bbc11ca61e67f79a5d7d3eed9f96be1e7552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-c, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:03 smithi161 podman[74736]: 2024-08-16 19:27:03.358230718 +0000 UTC m=+0.587332891 container cleanup 171681b2b1a444c98748c56e6380bbc11ca61e67f79a5d7d3eed9f96be1e7552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-16T19:27:03.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Aug 16 19:27:03 smithi161 bash[74736]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mon-c 2024-08-16T19:27:03.898 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mon.c.service' 2024-08-16T19:27:03.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:03.956 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-16T19:27:03.956 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-16T19:27:03.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x 2024-08-16T19:27:04.247 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:04 smithi062 systemd[1]: Stopping Ceph mgr.x for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:04.619 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:04 smithi062 podman[104421]: 2024-08-16 19:27:04.307085761 +0000 UTC m=+0.146486025 container died 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-16T19:27:04.987 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:04 smithi062 podman[104421]: 2024-08-16 19:27:04.718689606 +0000 UTC m=+0.558089866 container cleanup 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux ) 2024-08-16T19:27:04.988 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:04 smithi062 bash[104421]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x 2024-08-16T19:27:05.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-16T19:27:05.242 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:04 smithi062 podman[104436]: 2024-08-16 19:27:04.987422743 +0000 UTC m=+0.675305580 container remove 0d3d747f003f607c0c562be98827d24fa8a1ad5597104fa005d57a81b8832922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d863d21c9d1db9c3453c09dacd68766969720a44, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-16T19:27:05.243 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:05 smithi062 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service: Deactivated successfully. 2024-08-16T19:27:05.243 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:05 smithi062 systemd[1]: Stopped Ceph mgr.x for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:27:05.243 INFO:journalctl@ceph.mgr.x.smithi062.stdout:Aug 16 19:27:05 smithi062 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.x.service: Consumed 59.588s CPU time. 2024-08-16T19:27:05.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:05.259 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-16T19:27:05.259 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-16T19:27:05.260 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.y 2024-08-16T19:27:05.579 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:27:05 smithi155 systemd[1]: Stopping Ceph mgr.y for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:05.868 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:27:05 smithi155 podman[75590]: 2024-08-16 19:27:05.595171121 +0000 UTC m=+0.138717045 container died c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-16T19:27:06.261 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:27:05 smithi155 podman[75590]: 2024-08-16 19:27:05.967230764 +0000 UTC m=+0.510776688 container cleanup c4ffd3fc9417763e12375dfcfe64e0d24261f7508bbeea33ce61932a74935f1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-16T19:27:06.261 INFO:journalctl@ceph.mgr.y.smithi155.stdout:Aug 16 19:27:05 smithi155 bash[75590]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-y 2024-08-16T19:27:06.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-16T19:27:06.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:06.480 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-16T19:27:06.480 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-16T19:27:06.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.z 2024-08-16T19:27:06.826 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:27:06 smithi161 systemd[1]: Stopping Ceph mgr.z for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:07.081 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:27:06 smithi161 podman[74859]: 2024-08-16 19:27:06.825691579 +0000 UTC m=+0.143165112 container died 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-16T19:27:07.382 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:27:07 smithi161 podman[74859]: 2024-08-16 19:27:07.138499167 +0000 UTC m=+0.455972683 container cleanup 85dbe540ff065a41cc09566773fd79e9efd9aa61cdc9a4d27e0304d1c8ba1f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-16T19:27:07.382 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Aug 16 19:27:07 smithi161 bash[74859]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mgr-z 2024-08-16T19:27:07.533 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-16T19:27:07.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:07.566 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-16T19:27:07.567 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-16T19:27:07.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.0 2024-08-16T19:27:07.875 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:07 smithi062 systemd[1]: Stopping Ceph osd.0 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:08.246 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:27:07.873+0000 7fbe834df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:08.247 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:27:07.873+0000 7fbe834df640 -1 osd.0 117 *** Got signal Terminated *** 2024-08-16T19:27:08.247 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0[46984]: 2024-08-16T19:27:07.873+0000 7fbe834df640 -1 osd.0 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:13.567 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:13 smithi062 podman[104543]: 2024-08-16 19:27:13.266937513 +0000 UTC m=+5.498861083 container died a76aa2dc46aa90dcb92586a26d07e28d0a04fbc8c616f41cc31d5a7636d18409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:13.893 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:13 smithi062 podman[104543]: 2024-08-16 19:27:13.657403831 +0000 UTC m=+5.889327391 container cleanup a76aa2dc46aa90dcb92586a26d07e28d0a04fbc8c616f41cc31d5a7636d18409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-16T19:27:13.893 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:13 smithi062 bash[104543]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0 2024-08-16T19:27:14.246 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:13 smithi062 podman[104559]: 2024-08-16 19:27:13.892184678 +0000 UTC m=+0.627986256 container remove a76aa2dc46aa90dcb92586a26d07e28d0a04fbc8c616f41cc31d5a7636d18409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-16T19:27:14.496 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.148531537 +0000 UTC m=+0.017687932 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:14.496 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.251806288 +0000 UTC m=+0.120962663 container create 7195701f98a73373cd815fed4584a005795b9a7468c5deb0e0bb71e86ba27cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-08-16T19:27:14.747 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.496277311 +0000 UTC m=+0.365433687 container init 7195701f98a73373cd815fed4584a005795b9a7468c5deb0e0bb71e86ba27cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21) 2024-08-16T19:27:14.747 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.500766227 +0000 UTC m=+0.369922602 container start 7195701f98a73373cd815fed4584a005795b9a7468c5deb0e0bb71e86ba27cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-16T19:27:14.747 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.596342722 +0000 UTC m=+0.465499110 container attach 7195701f98a73373cd815fed4584a005795b9a7468c5deb0e0bb71e86ba27cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:27:14.747 INFO:journalctl@ceph.osd.0.smithi062.stdout:Aug 16 19:27:14 smithi062 podman[104649]: 2024-08-16 19:27:14.702307203 +0000 UTC m=+0.571463588 container died 7195701f98a73373cd815fed4584a005795b9a7468c5deb0e0bb71e86ba27cf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-0-deactivate, RELEASE=squid-d863d21, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:15.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.0.service' 2024-08-16T19:27:15.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:15.412 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-16T19:27:15.413 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-16T19:27:15.413 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.1 2024-08-16T19:27:15.730 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:15 smithi155 systemd[1]: Stopping Ceph osd.1 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:16.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:27:15.729+0000 7f07c5a92640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:16.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:27:15.729+0000 7f07c5a92640 -1 osd.1 117 *** Got signal Terminated *** 2024-08-16T19:27:16.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1[41765]: 2024-08-16T19:27:15.729+0000 7f07c5a92640 -1 osd.1 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:21.466 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:21 smithi155 podman[75716]: 2024-08-16 19:27:21.168358618 +0000 UTC m=+5.557381772 container died 2a209a23697472685be07298fb1d75c6dfd9b2d802a5061f65b1eb127b915901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:21.749 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:21 smithi155 podman[75716]: 2024-08-16 19:27:21.564338057 +0000 UTC m=+5.953361210 container cleanup 2a209a23697472685be07298fb1d75c6dfd9b2d802a5061f65b1eb127b915901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-08-16T19:27:21.750 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:21 smithi155 bash[75716]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1 2024-08-16T19:27:22.079 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:21 smithi155 podman[75731]: 2024-08-16 19:27:21.749342225 +0000 UTC m=+0.579533941 container remove 2a209a23697472685be07298fb1d75c6dfd9b2d802a5061f65b1eb127b915901 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21) 2024-08-16T19:27:22.445 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.087314521 +0000 UTC m=+0.017752661 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:22.446 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.192134898 +0000 UTC m=+0.122573085 container create 92ef415ee224c3646678cfdb7f1c6f5ce03301d9f3d5d2a18f3975d99e2e2411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-16T19:27:22.829 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.445433466 +0000 UTC m=+0.375871612 container init 92ef415ee224c3646678cfdb7f1c6f5ce03301d9f3d5d2a18f3975d99e2e2411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-16T19:27:22.830 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.448835748 +0000 UTC m=+0.379273892 container start 92ef415ee224c3646678cfdb7f1c6f5ce03301d9f3d5d2a18f3975d99e2e2411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:22.830 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.503657701 +0000 UTC m=+0.434095861 container attach 92ef415ee224c3646678cfdb7f1c6f5ce03301d9f3d5d2a18f3975d99e2e2411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:27:22.830 INFO:journalctl@ceph.osd.1.smithi155.stdout:Aug 16 19:27:22 smithi155 podman[75822]: 2024-08-16 19:27:22.676793351 +0000 UTC m=+0.607231493 container died 92ef415ee224c3646678cfdb7f1c6f5ce03301d9f3d5d2a18f3975d99e2e2411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:23.297 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.1.service' 2024-08-16T19:27:23.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:23.336 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-16T19:27:23.336 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-16T19:27:23.336 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.2 2024-08-16T19:27:23.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:23 smithi161 systemd[1]: Stopping Ceph osd.2 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:24.388 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:27:23.917+0000 7fd308d6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:24.388 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:27:23.917+0000 7fd308d6b640 -1 osd.2 117 *** Got signal Terminated *** 2024-08-16T19:27:24.388 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2[42540]: 2024-08-16T19:27:23.917+0000 7fd308d6b640 -1 osd.2 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:29.638 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:29 smithi161 podman[74987]: 2024-08-16 19:27:29.381408404 +0000 UTC m=+5.852077740 container died 7f674a498cc046dfbf740e7d019472a1f4e6d46e9c2507dd27b5d24ae96bc389 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:27:30.053 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:29 smithi161 podman[74987]: 2024-08-16 19:27:29.776589857 +0000 UTC m=+6.247259182 container cleanup 7f674a498cc046dfbf740e7d019472a1f4e6d46e9c2507dd27b5d24ae96bc389 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21) 2024-08-16T19:27:30.053 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:29 smithi161 bash[74987]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2 2024-08-16T19:27:30.373 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75014]: 2024-08-16 19:27:30.053387653 +0000 UTC m=+0.669751496 container remove 7f674a498cc046dfbf740e7d019472a1f4e6d46e9c2507dd27b5d24ae96bc389 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-16T19:27:30.632 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.275697027 +0000 UTC m=+0.017628178 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:30.632 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.387864776 +0000 UTC m=+0.129795972 container create 9226cafdab9bc5c7d4b5978c3febfffd5887be8d9cfe11a2e627312cae19e4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=) 2024-08-16T19:27:30.632 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.631982443 +0000 UTC m=+0.373913584 container init 9226cafdab9bc5c7d4b5978c3febfffd5887be8d9cfe11a2e627312cae19e4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-16T19:27:30.888 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.635305681 +0000 UTC m=+0.377236822 container start 9226cafdab9bc5c7d4b5978c3febfffd5887be8d9cfe11a2e627312cae19e4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, io.buildah.version=1.36.0) 2024-08-16T19:27:30.915 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.723752467 +0000 UTC m=+0.465683609 container attach 9226cafdab9bc5c7d4b5978c3febfffd5887be8d9cfe11a2e627312cae19e4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-16T19:27:30.916 INFO:journalctl@ceph.osd.2.smithi161.stdout:Aug 16 19:27:30 smithi161 podman[75108]: 2024-08-16 19:27:30.822722496 +0000 UTC m=+0.564653642 container died 9226cafdab9bc5c7d4b5978c3febfffd5887be8d9cfe11a2e627312cae19e4db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=) 2024-08-16T19:27:31.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.2.service' 2024-08-16T19:27:31.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:31.535 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-16T19:27:31.535 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-16T19:27:31.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.3 2024-08-16T19:27:31.846 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:31 smithi062 systemd[1]: Stopping Ceph osd.3 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:32.124 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:27:31.845+0000 7f75f86d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:32.124 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:27:31.845+0000 7f75f86d2640 -1 osd.3 117 *** Got signal Terminated *** 2024-08-16T19:27:32.124 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:31 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:27:31.845+0000 7f75f86d2640 -1 osd.3 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:34.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:33.638+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:34.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:34.309+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:35.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:34.660+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:35.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:34 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:34.973+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:35.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:35.305+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:35.996 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:35 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:27:35.733+0000 7f75f4ce8640 -1 osd.3 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:09.602142+0000 front 2024-08-16T19:27:09.602108+0000 (oldest deadline 2024-08-16T19:27:34.802957+0000) 2024-08-16T19:27:36.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:35.691+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:36.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:35 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:35.887+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:36.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:36.005+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:36.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:36.323+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:37.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:36.713+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:37.227 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3[53408]: 2024-08-16T19:27:36.778+0000 7f75f4ce8640 -1 osd.3 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:09.602142+0000 front 2024-08-16T19:27:09.602108+0000 (oldest deadline 2024-08-16T19:27:34.802957+0000) 2024-08-16T19:27:37.227 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:36 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:36.981+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:37.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:36 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:36.919+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:37.496 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:37 smithi062 podman[104765]: 2024-08-16 19:27:37.226701722 +0000 UTC m=+5.485338962 container died df3808467e7e8e15b344a31b60b06bcf8fe487ba14262fad84977418a2fead29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:37.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:37.277+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:37.855 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:37 smithi062 podman[104765]: 2024-08-16 19:27:37.620672573 +0000 UTC m=+5.879309858 container cleanup df3808467e7e8e15b344a31b60b06bcf8fe487ba14262fad84977418a2fead29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-d863d21, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-16T19:27:37.855 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:37 smithi062 bash[104765]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3 2024-08-16T19:27:38.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:37.670+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:38.199 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:37 smithi062 podman[104793]: 2024-08-16 19:27:37.855165127 +0000 UTC m=+0.626678715 container remove df3808467e7e8e15b344a31b60b06bcf8fe487ba14262fad84977418a2fead29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-d863d21, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-16T19:27:38.200 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:37 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:37.944+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:38.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:37 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:37.909+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:38.460 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.101764459 +0000 UTC m=+0.018159898 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:38.460 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.214777788 +0000 UTC m=+0.131173262 container create d7b0028ba394ed52ede8eb1ba188164bc2c662bef2ad3121cba7023d7a8bc403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:27:38.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:38.253+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:38.747 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.459554691 +0000 UTC m=+0.375950139 container init d7b0028ba394ed52ede8eb1ba188164bc2c662bef2ad3121cba7023d7a8bc403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2) 2024-08-16T19:27:38.747 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.463238643 +0000 UTC m=+0.379634072 container start d7b0028ba394ed52ede8eb1ba188164bc2c662bef2ad3121cba7023d7a8bc403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-16T19:27:38.747 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.517725294 +0000 UTC m=+0.434120734 container attach d7b0028ba394ed52ede8eb1ba188164bc2c662bef2ad3121cba7023d7a8bc403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:38.747 INFO:journalctl@ceph.osd.3.smithi062.stdout:Aug 16 19:27:38 smithi062 podman[104885]: 2024-08-16 19:27:38.660447211 +0000 UTC m=+0.576842642 container died d7b0028ba394ed52ede8eb1ba188164bc2c662bef2ad3121cba7023d7a8bc403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-08-16T19:27:39.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:38.643+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:39.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:38.794+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:39.238 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:38 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:38.961+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:39.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.3.service' 2024-08-16T19:27:39.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:39.296 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-16T19:27:39.297 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-16T19:27:39.297 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.4 2024-08-16T19:27:39.323 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:38 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:38.908+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:39.579 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:39 smithi155 systemd[1]: Stopping Ceph osd.4 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:39.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:39.256+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:39.888 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:39.757+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:39.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:39.608+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:39.903 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:39 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:27:39.593+0000 7f35e0122640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:39.904 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:39 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:27:39.593+0000 7f35e0122640 -1 osd.4 117 *** Got signal Terminated *** 2024-08-16T19:27:39.904 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:39 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:27:39.593+0000 7f35e0122640 -1 osd.4 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:40.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:39 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:39.962+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:40.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:39 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:39.903+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:40.636 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:40 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:40.298+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:40.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:40 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:40.636+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:40.888 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:40 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:40.753+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:41.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:40 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:40.947+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:41.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:40 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:40.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:41.546 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:41 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:41.264+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:41.888 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:41 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:41.725+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:41.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:41 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:41.684+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:42.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:41 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:41.968+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:42.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:42 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:42.015+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:42.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:42 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:42.255+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:43.079 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:42 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:27:42.736+0000 7f35d8730640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.853050+0000 front 2024-08-16T19:27:18.853088+0000 (oldest deadline 2024-08-16T19:27:41.753174+0000) 2024-08-16T19:27:43.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:42 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:42.924+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:43.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:42 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:42.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:43.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:42 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:42.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:43.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:42 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:42.718+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:43.139 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:42 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:42.718+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:43.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:42 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:42.980+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:43.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:42 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:42.980+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:43.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:43 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:43.223+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:44.079 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:43 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4[48624]: 2024-08-16T19:27:43.776+0000 7f35d8730640 -1 osd.4 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.853050+0000 front 2024-08-16T19:27:18.853088+0000 (oldest deadline 2024-08-16T19:27:41.753174+0000) 2024-08-16T19:27:44.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:43 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:43.967+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:44.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:43 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:43.967+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:44.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:43 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:43.751+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:44.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:43 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:43.751+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:44.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:43 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:43.718+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:44.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:43 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:43.718+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:44.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:43 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:43.946+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:44.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:43 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:43.946+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:44.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:44 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:44.220+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:44.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:44 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:44.454+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:45.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:44 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:44.789+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:45.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:44 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:44.789+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:45.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:44 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:44.731+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:45.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:44 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:44.731+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:45.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:44 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:44.939+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:45.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:44 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:44.939+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:45.329 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:45 smithi155 podman[75935]: 2024-08-16 19:27:45.042378574 +0000 UTC m=+5.553394980 container died cfefa2060a92a6b299621a709933e2593c20bfb61977419925c46ca57ca279da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux ) 2024-08-16T19:27:45.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:44 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:44.942+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:45.330 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:44 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:44.942+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:45.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:45.248+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:45.712 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:45 smithi155 podman[75935]: 2024-08-16 19:27:45.419359885 +0000 UTC m=+5.930376283 container cleanup cfefa2060a92a6b299621a709933e2593c20bfb61977419925c46ca57ca279da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0) 2024-08-16T19:27:45.713 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:45 smithi155 bash[75935]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4 2024-08-16T19:27:45.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:45 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:45.488+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:46.038 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:45 smithi155 podman[75953]: 2024-08-16 19:27:45.712496209 +0000 UTC m=+0.663860184 container remove cfefa2060a92a6b299621a709933e2593c20bfb61977419925c46ca57ca279da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:46.039 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:45 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:45.902+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:46.039 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:45 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:45.902+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:46.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:45.808+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:46.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:45.808+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:46.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:45.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:46.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:45 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:45.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:46.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:45 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:45.955+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:46.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:45 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:45.955+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:46.292 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:46.038417035 +0000 UTC m=+0.112788959 container create 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-16T19:27:46.292 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:45.9431815 +0000 UTC m=+0.017553428 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:46.579 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:46.291403335 +0000 UTC m=+0.365775274 container init 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:27:46.579 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:46.295197823 +0000 UTC m=+0.369569748 container start 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-16T19:27:46.579 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:46.358117205 +0000 UTC m=+0.432489142 container attach 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-16T19:27:46.579 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:46 smithi155 podman[76040]: 2024-08-16 19:27:46.48329666 +0000 UTC m=+0.557668589 container died 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-16T19:27:46.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:46 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:46.236+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:46.913 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:46 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:46.620+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:46.945 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:46 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:46.534+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:47.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.4.service' 2024-08-16T19:27:47.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:46 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:46.786+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:47.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:46 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:46.786+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:47.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:46 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:46.678+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:47.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:46 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:46.678+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:47.173 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:46 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:46.912+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:47.173 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:46 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:46.912+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:47.173 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:47 smithi155 podman[76064]: 2024-08-16 19:27:47.095631308 +0000 UTC m=+0.603072421 container remove 12a948cd50b79bdf54924c8428c2443ea8d2fe224a6fb760a2e8d3fcb2488c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-16T19:27:47.173 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:47 smithi155 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.4.service: Deactivated successfully. 2024-08-16T19:27:47.173 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:47 smithi155 systemd[1]: Stopped Ceph osd.4 for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:27:47.174 INFO:journalctl@ceph.osd.4.smithi155.stdout:Aug 16 19:27:47 smithi155 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.4.service: Consumed 2min 10.017s CPU time. 2024-08-16T19:27:47.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:47.188 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-16T19:27:47.188 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-16T19:27:47.188 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.5 2024-08-16T19:27:47.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:46 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:46.944+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:47.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:46 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:46.944+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:47.473 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 systemd[1]: Stopping Ceph osd.5 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:47.474 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:47.248+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:47.818 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:47.652+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:47.819 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:47.652+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:47.819 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:47.472+0000 7f09f966f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:47.819 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:47.472+0000 7f09f966f640 -1 osd.5 117 *** Got signal Terminated *** 2024-08-16T19:27:47.819 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:47.472+0000 7f09f966f640 -1 osd.5 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:47.819 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:47.818+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:47.820 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:47 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:47.818+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:47.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:47 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:47.573+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:47.895 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:47 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:47.505+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:48.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:47 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:47.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:48.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:47 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:47.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:48.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:47 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:47.877+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:48.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:47 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:47.877+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:48.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:48 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:48.201+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:48.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:48 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:48.554+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:48.847 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:48.509+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:49.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:48 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:48.789+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:49.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:48 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:48.789+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:49.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:48 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:48.643+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:49.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:48 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:48.643+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:49.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:48.846+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:49.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:48.846+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:49.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:48 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:48.888+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:49.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:48 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:48.888+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:49.607 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:49 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:49.168+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:49.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:49 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:49.490+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:49.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:49 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:49.565+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:49.888 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:49 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:49.833+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:49.889 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:49 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:49.833+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:49.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:49 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:49.606+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:49.890 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:49 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:49.606+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:50.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:49 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:49.840+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:50.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:49 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:49.840+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:50.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:49 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:49.843+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:50.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:49 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:49.843+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:50.605 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:50.179+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:50.825 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:50.506+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:50.856 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:50.605+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:50.857 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:50.605+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:50.868 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:50.615+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:51.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:50.856+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:51.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:50 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:50.856+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:51.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:50.825+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:51.363 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:50 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:50.825+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:51.363 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:50.868+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:51.363 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:50 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:50.868+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:51.600 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:51 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:51.566+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:51.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:51 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:51.170+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:51.791 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:51 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:51.527+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:51.791 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:51 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:51.527+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:52.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:51 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:51.829+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:52.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:51 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:51.829+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:52.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:51 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:51.829+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:52.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:51 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:51.874+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.211375+0000 front 2024-08-16T19:27:12.211344+0000 (oldest deadline 2024-08-16T19:27:38.111217+0000) 2024-08-16T19:27:52.138 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:51 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5[48996]: 2024-08-16T19:27:51.874+0000 7f09f5c85640 -1 osd.5 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:20.412421+0000 front 2024-08-16T19:27:20.412486+0000 (oldest deadline 2024-08-16T19:27:42.013351+0000) 2024-08-16T19:27:52.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:51 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:51.641+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:52.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:51 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:51.641+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:52.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:51 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:51.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:52.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:51 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:51.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:52.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:52.217+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:52.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:52.217+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:52.742 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:52.487+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:52.742 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:52.487+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:52.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:52 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:52.596+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:52.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:52 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:52.596+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:52.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:52.742+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:52.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:52 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:52.742+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:53.139 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:52 smithi161 podman[75211]: 2024-08-16 19:27:52.910336645 +0000 UTC m=+5.521755769 container died de4944b9a03e6dffd84dba9aab3ecb22e82f361b9a5c97b1bf1d16ab701d4bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-08-16T19:27:53.140 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:52.679+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:53.140 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:52.679+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:53.140 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:52 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:52.679+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:53.331 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:52 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:52.844+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:53.332 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:52 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:52.844+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:53.332 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:52 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:52.844+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:53.549 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:53 smithi161 podman[75211]: 2024-08-16 19:27:53.263834367 +0000 UTC m=+5.875253480 container cleanup de4944b9a03e6dffd84dba9aab3ecb22e82f361b9a5c97b1bf1d16ab701d4bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-08-16T19:27:53.549 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:53 smithi161 bash[75211]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5 2024-08-16T19:27:53.549 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:53.267+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:53.549 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:53.267+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:53.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:53.468+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:53.747 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:53.468+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:53.747 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:53.700+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:53.747 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:53.700+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:53.747 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:53.700+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:53.841 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:53 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:53.587+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:53.841 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:53 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:53.587+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:53.888 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:53 smithi161 podman[75226]: 2024-08-16 19:27:53.548451486 +0000 UTC m=+0.632797038 container remove de4944b9a03e6dffd84dba9aab3ecb22e82f361b9a5c97b1bf1d16ab701d4bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-16T19:27:53.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:53.641+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:53.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:53.641+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:53.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:53 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:53.641+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:54.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:53 smithi161 podman[75314]: 2024-08-16 19:27:53.810643403 +0000 UTC m=+0.016962867 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:27:54.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:53 smithi161 podman[75314]: 2024-08-16 19:27:53.916469537 +0000 UTC m=+0.122788981 container create ed10508a7fadacf42727b29dd82ca1ace3fe13fba82a62b1fd4ecb09b8879457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=) 2024-08-16T19:27:54.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:54 smithi161 podman[75314]: 2024-08-16 19:27:54.127050397 +0000 UTC m=+0.333369845 container init ed10508a7fadacf42727b29dd82ca1ace3fe13fba82a62b1fd4ecb09b8879457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-08-16T19:27:54.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:54 smithi161 podman[75314]: 2024-08-16 19:27:54.13032361 +0000 UTC m=+0.336643056 container start ed10508a7fadacf42727b29dd82ca1ace3fe13fba82a62b1fd4ecb09b8879457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-16T19:27:54.194 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:54 smithi161 podman[75314]: 2024-08-16 19:27:54.193706834 +0000 UTC m=+0.400026290 container attach ed10508a7fadacf42727b29dd82ca1ace3fe13fba82a62b1fd4ecb09b8879457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-16T19:27:54.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:53 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:53.840+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:54.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:53 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:53.840+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:54.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:53 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:53.840+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:54.638 INFO:journalctl@ceph.osd.5.smithi161.stdout:Aug 16 19:27:54 smithi161 podman[75314]: 2024-08-16 19:27:54.316312576 +0000 UTC m=+0.522632022 container died ed10508a7fadacf42727b29dd82ca1ace3fe13fba82a62b1fd4ecb09b8879457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21) 2024-08-16T19:27:54.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:54.237+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:54.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:54.237+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:54.712 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:54.450+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:54.712 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:54.450+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:54.956 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:54.678+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:54.956 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:54.678+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:54.956 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:54 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:54.678+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:54.971 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.5.service' 2024-08-16T19:27:54.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:54.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:54.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:54.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:54.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:54 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:54.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:55.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:27:55.010 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-16T19:27:55.010 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-16T19:27:55.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.6 2024-08-16T19:27:55.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:54.597+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:55.080 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:54.597+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:55.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:54.833+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:55.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:54.833+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:55.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:54 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:54.833+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:55.341 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 systemd[1]: Stopping Ceph osd.6 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:27:55.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:55.285+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:55.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:55.285+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:55.712 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:55.340+0000 7ff51243a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:27:55.713 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:55.340+0000 7ff51243a640 -1 osd.6 117 *** Got signal Terminated *** 2024-08-16T19:27:55.713 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:55.340+0000 7ff51243a640 -1 osd.6 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:27:55.713 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:55.485+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:55.713 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:55.485+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:55.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:55.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:55.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:55.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:55.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:55.712+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:56.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:55.644+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:56.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:55.644+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:56.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:55.843+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:56.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:55.843+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:56.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:55 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:55.843+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:56.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:55.719+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:56.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:55.719+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:56.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:55 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:55.719+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:56.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:56.240+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:56.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:56.240+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:56.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:56.462+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:56.747 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:56.462+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:57.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:56.686+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:57.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:56.686+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:57.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:56.854+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:57.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:56.854+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:57.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:56 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:56.854+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:57.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:56.763+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:57.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:56.763+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:57.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:56 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:56.763+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:57.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:56.748+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:57.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:56.748+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:57.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:56 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:56.748+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:57.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:57.289+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:57.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:57.289+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:57.732 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:57.414+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:57.732 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:57.414+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:57.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:57.731+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:57.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:57.731+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:57.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:57 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:57.731+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:58.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:57.693+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:58.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:57.693+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:58.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:57.807+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:58.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:57.807+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:58.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:57.807+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:58.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:57 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:57.807+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:27:58.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:57.745+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:58.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:57.745+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:58.139 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:57 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:57.745+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:58.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:58.243+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:58.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:58.243+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:58.737 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:58 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:58.420+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:58.737 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:58 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:58.420+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:27:58.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:58 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:58.737+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:27:58.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:58 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:58.737+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:27:58.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:58 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:58.737+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:27:59.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:58.717+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:27:59.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:58.717+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:27:59.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:58.785+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:27:59.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:58.785+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:27:59.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:58.785+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:27:59.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:58.785+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:27:59.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:58.746+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:27:59.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:58.746+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:27:59.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:58.746+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:27:59.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:58 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:58.746+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:27:59.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:59.199+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:27:59.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:27:59.199+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:27:59.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:59 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:59.466+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:18.833976+0000 front 2024-08-16T19:27:18.833937+0000 (oldest deadline 2024-08-16T19:27:43.534013+0000) 2024-08-16T19:27:59.747 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:27:59 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6[59835]: 2024-08-16T19:27:59.466+0000 7ff50aa48640 -1 osd.6 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.235397+0000 front 2024-08-16T19:27:25.235493+0000 (oldest deadline 2024-08-16T19:27:50.535498+0000) 2024-08-16T19:28:00.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:59.710+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:00.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:59.710+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:00.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:27:59.710+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:00.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:59.754+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:00.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:59.754+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:00.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:59.754+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:00.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:27:59 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:27:59.754+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:00.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:59.723+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:00.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:59.723+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:00.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:59.723+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:00.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:27:59 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:27:59.723+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:00.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:59 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:59.773+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:00.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:59 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:59.773+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:00.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:27:59 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:27:59.773+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:00.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:00.239+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:00.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:00.239+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:01.058 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:00 smithi062 podman[104988]: 2024-08-16 19:28:00.790786077 +0000 UTC m=+5.564857882 container died 101ea39e160b696daf13827f351ed3eab86acef5641f8c111acad9d30880661e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-08-16T19:28:01.058 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:00.765+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:01.059 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:00.765+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:01.059 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:00.765+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:01.059 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:00 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:00.765+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:01.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:00.671+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:01.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:00.671+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:01.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:00.671+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:01.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:00.705+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:01.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:00.705+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:01.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:00.705+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:01.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:00 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:00.705+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:01.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:00.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:01.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:00.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:01.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:00.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:01.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:00 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:00.724+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:01.417 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:01 smithi062 podman[104988]: 2024-08-16 19:28:01.149112588 +0000 UTC m=+5.923184384 container cleanup 101ea39e160b696daf13827f351ed3eab86acef5641f8c111acad9d30880661e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:28:01.417 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:01 smithi062 bash[104988]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6 2024-08-16T19:28:01.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:01.286+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:01.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:01.287+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:01.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:01.287+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:01.746 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:01 smithi062 podman[105003]: 2024-08-16 19:28:01.417107665 +0000 UTC m=+0.621028986 container remove 101ea39e160b696daf13827f351ed3eab86acef5641f8c111acad9d30880661e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, ceph=True, org.label-schema.schema-version=1.0) 2024-08-16T19:28:02.034 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:01.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:02.034 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:01.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:02.034 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:01.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:02.034 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:01.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:02.035 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:01.667+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:02.035 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:01.667+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:02.035 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:01 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:01.667+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:02.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:01.711+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:02.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:01.711+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:02.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:01.711+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:02.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:01 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:01.711+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:02.188 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:01 smithi062 podman[105095]: 2024-08-16 19:28:01.693681289 +0000 UTC m=+0.017721136 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:02.188 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:01 smithi062 podman[105095]: 2024-08-16 19:28:01.851578155 +0000 UTC m=+0.175617990 container create 4973eb894831e373b7ecae93919347344bfdc543c3a216cca34b4ae265f30997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-16T19:28:02.188 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:01 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:01.809+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:02.189 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:01 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:01.809+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:02.189 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:01 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:01.809+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:02.189 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:01 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:01.809+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:02.497 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:02 smithi062 podman[105095]: 2024-08-16 19:28:02.188068482 +0000 UTC m=+0.512108317 container init 4973eb894831e373b7ecae93919347344bfdc543c3a216cca34b4ae265f30997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:02.497 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:02 smithi062 podman[105095]: 2024-08-16 19:28:02.191872176 +0000 UTC m=+0.515912011 container start 4973eb894831e373b7ecae93919347344bfdc543c3a216cca34b4ae265f30997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:28:02.497 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:02 smithi062 podman[105095]: 2024-08-16 19:28:02.278990753 +0000 UTC m=+0.603030598 container attach 4973eb894831e373b7ecae93919347344bfdc543c3a216cca34b4ae265f30997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-16T19:28:02.497 INFO:journalctl@ceph.osd.6.smithi062.stdout:Aug 16 19:28:02 smithi062 podman[105095]: 2024-08-16 19:28:02.382516648 +0000 UTC m=+0.706556496 container died 4973eb894831e373b7ecae93919347344bfdc543c3a216cca34b4ae265f30997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-16T19:28:02.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:02.253+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:02.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:02.253+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:02.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:02.253+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:02.991 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:02 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:02.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:02.992 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:02 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:02.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:02.992 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:02 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:02.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:02.992 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:02 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:02.790+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:03.012 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.6.service' 2024-08-16T19:28:03.052 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:03.053 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-16T19:28:03.053 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-16T19:28:03.053 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.7 2024-08-16T19:28:03.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:02.686+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:03.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:02.686+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:03.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:02.686+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:03.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:02.684+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:03.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:02.684+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:03.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:02.684+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:03.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:02 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:02.684+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:03.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:02.685+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:03.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:02.685+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:03.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:02.685+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:03.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:02 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:02.685+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:03.374 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 systemd[1]: Stopping Ceph osd.7 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:03.374 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.372+0000 7fdf661ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:03.637 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:03.259+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:03.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:03.259+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:03.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:03.259+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:03.640 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.372+0000 7fdf661ad640 -1 osd.7 117 *** Got signal Terminated *** 2024-08-16T19:28:03.640 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.372+0000 7fdf661ad640 -1 osd.7 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:28:03.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:03.636+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:03.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:03.636+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:03.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:03.636+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:03.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:03 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:03.636+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:03.996 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:03 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:03.827+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:03.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:03 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:03.827+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:03.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:03 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:03.827+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:03.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:03 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:03.827+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:04.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.668+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:04.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.668+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:04.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:03.668+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:04.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:03.639+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:04.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:03.639+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:04.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:03.639+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:04.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:03.639+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:04.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:04.237+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:04.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:04.237+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:04.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:04.237+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:05.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:04.693+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:05.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:04.693+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:05.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:04.693+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:05.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:04.600+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:05.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:04.600+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:05.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:04.600+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:05.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:04 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:04.600+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:05.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:04.662+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:05.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:04.662+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:05.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:04.662+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:05.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:04 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:04.662+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:05.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:04 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:04.807+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:05.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:04 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:04.807+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:05.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:04 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:04.807+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:05.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:04 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:04.807+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:05.615 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:05.191+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:05.615 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:05.191+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:05.615 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:05.191+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:05.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:05.614+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:05.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:05.614+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:05.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:05.614+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:05.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:05 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:05.614+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:06.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:05.680+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:06.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:05.680+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:06.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:05.680+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:06.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:05.607+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:06.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:05.607+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:06.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:05.607+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:06.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:05 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:05.607+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:06.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:05 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:05.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:06.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:05 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:05.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:06.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:05 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:05.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:06.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:05 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:05.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:06.588 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:06.162+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:06.589 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:06.162+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:06.589 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:06.162+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:06.589 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:06.162+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:06.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:06.587+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:06.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:06.587+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:06.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:06.587+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:06.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:06 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:06.587+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:07.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:06.658+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:07.079 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:06.658+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:07.080 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:06.658+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:07.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:06.591+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:07.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:06.591+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:07.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:06.591+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:07.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:06 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:06.591+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:07.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:06 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:06.808+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:07.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:06 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:06.808+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:07.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:06 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:06.808+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:07.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:06 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:06.808+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:07.596 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:07.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:07.596 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:07.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:07.597 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:07.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:07.597 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:07.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:07.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:07.638+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:19.079999+0000 front 2024-08-16T19:27:19.079962+0000 (oldest deadline 2024-08-16T19:27:46.080888+0000) 2024-08-16T19:28:07.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:07.638+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.782069+0000 front 2024-08-16T19:27:27.781855+0000 (oldest deadline 2024-08-16T19:27:51.882025+0000) 2024-08-16T19:28:07.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:07.638+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.783061+0000 front 2024-08-16T19:27:34.783129+0000 (oldest deadline 2024-08-16T19:27:59.483073+0000) 2024-08-16T19:28:07.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7[55568]: 2024-08-16T19:28:07.638+0000 7fdf627c3640 -1 osd.7 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:41.184039+0000 front 2024-08-16T19:27:41.184014+0000 (oldest deadline 2024-08-16T19:28:07.084220+0000) 2024-08-16T19:28:07.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:07.570+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:07.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:07.570+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:07.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:07.570+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:07.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:07 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:07.570+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:07.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:07.595+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:07.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:07.595+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:07.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:07.595+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:07.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:07 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:07.595+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:08.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:07.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:08.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:07.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:08.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:07.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:08.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:07 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:07.856+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:08.563 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:08.168+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:08.563 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:08.168+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:08.563 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:08.168+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:08.564 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:08.168+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:08.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:08.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:08.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:08.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:08.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:08.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:08.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:08 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:08.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:08.902 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:08.602+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:08.902 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:08.602+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:08.902 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:08.602+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:08.902 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:08.602+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:09.175 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:08 smithi155 podman[76156]: 2024-08-16 19:28:08.901805973 +0000 UTC m=+5.644942611 container died 1b71f96fc906b1498255a6363acc7a70964db0d43cacf83739718fc781de0146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-16T19:28:09.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:08 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:08.891+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:09.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:08 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:08.891+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:09.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:08 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:08.891+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:09.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:08 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:08.891+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:09.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:08 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:08.891+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:09.572 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:09 smithi155 podman[76156]: 2024-08-16 19:28:09.265142273 +0000 UTC m=+6.008278909 container cleanup 1b71f96fc906b1498255a6363acc7a70964db0d43cacf83739718fc781de0146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-16T19:28:09.572 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:09 smithi155 bash[76156]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7 2024-08-16T19:28:09.572 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:09.183+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:09.573 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:09.183+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:09.573 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:09.183+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:09.573 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:09.183+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:09.829 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:09 smithi155 podman[76173]: 2024-08-16 19:28:09.566600725 +0000 UTC m=+0.662864360 container remove 1b71f96fc906b1498255a6363acc7a70964db0d43cacf83739718fc781de0146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:28:09.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:09 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:09.632+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:09.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:09 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:09.632+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:09.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:09 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:09.632+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:09.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:09 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:09.632+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:09.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:09 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:09.632+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:09.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:09.570+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:09.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:09.570+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:09.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:09.570+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:09.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:09 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:09.570+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:10.213 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:09 smithi155 podman[76260]: 2024-08-16 19:28:09.789855357 +0000 UTC m=+0.016780939 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:10.213 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:09 smithi155 podman[76260]: 2024-08-16 19:28:09.909379982 +0000 UTC m=+0.136305556 container create 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-16T19:28:10.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:10 smithi155 podman[76260]: 2024-08-16 19:28:10.112117071 +0000 UTC m=+0.339042649 container init 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-16T19:28:10.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:10 smithi155 podman[76260]: 2024-08-16 19:28:10.115215473 +0000 UTC m=+0.342141058 container start 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, RELEASE=squid-d863d21, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-16T19:28:10.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:10 smithi155 podman[76260]: 2024-08-16 19:28:10.212146904 +0000 UTC m=+0.439072492 container attach 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-16T19:28:10.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:09 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:09.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:10.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:09 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:09.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:10.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:09 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:09.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:10.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:09 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:09.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:10.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:09 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:09.895+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:10.559 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:10.180+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:10.560 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:10.180+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:10.560 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:10.180+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:10.560 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:10.180+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:10.579 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:10 smithi155 podman[76260]: 2024-08-16 19:28:10.301676127 +0000 UTC m=+0.528601705 container died 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, org.label-schema.build-date=20240731) 2024-08-16T19:28:10.851 INFO:journalctl@ceph.osd.7.smithi155.stdout:Aug 16 19:28:10 smithi155 podman[76283]: 2024-08-16 19:28:10.83288382 +0000 UTC m=+0.525728495 container remove 547b6afe12b0c2f797f80511ee14992f45fb8ec498cfdebf0394287e93b205d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:10.852 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:10.652+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:10.852 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:10.652+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:10.852 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:10.652+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:10.852 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:10.652+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:10.852 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:10 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:10.652+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:10.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.7.service' 2024-08-16T19:28:10.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:10.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:10.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:10.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:10.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:10.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:10.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:10 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:10.559+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:10.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:10.934 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-16T19:28:10.934 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-16T19:28:10.934 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.8 2024-08-16T19:28:11.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:10 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:10.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:11.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:10 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:10.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:11.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:10 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:10.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:11.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:10 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:10.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:11.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:10 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:10.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:11.388 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:10 smithi161 systemd[1]: Stopping Ceph osd.8 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:11.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:11.134+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:11.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:11.134+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:11.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:11.134+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:11.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:11.134+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:11.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.500+0000 7fa26f976640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:11.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.500+0000 7fa26f976640 -1 osd.8 117 *** Got signal Terminated *** 2024-08-16T19:28:11.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.500+0000 7fa26f976640 -1 osd.8 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:28:11.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.601+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:11.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.601+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:11.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.601+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:11.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:11 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:11.601+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:11.989 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:11.662+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:11.989 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:11.662+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:11.989 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:11.662+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:11.990 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:11.662+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:11.990 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:11.662+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:12.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:11 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:11.943+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:12.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:11 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:11.943+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:12.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:11 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:11.943+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:12.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:11 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:11.943+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:12.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:11 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:11.943+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:12.634 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:12.171+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:12.635 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:12.171+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:12.635 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:12.171+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:12.635 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:12.171+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:12.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:12.634+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:12.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:12.634+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:12.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:12.634+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:12.889 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:12 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:12.634+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:13.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:12 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:12.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:13.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:12 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:12.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:13.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:12 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:12.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:13.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:12 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:12.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:13.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:12 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:12.659+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:13.246 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:12.923+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:13.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:12.923+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:13.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:12.923+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:13.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:12.923+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:13.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:12 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:12.923+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:13.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:13.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:13.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:13.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:13.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:13.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:13.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:13.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:13.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:13.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:13.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:13.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:13.997 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:13.998 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:13 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:13.956+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:14.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:14.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:14.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:14.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:14.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:14.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:13 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:13.626+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:14.083 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:13.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:14.083 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:13.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:14.083 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:13.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:14.083 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:13 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:13.668+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:14.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:14.082+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:14.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:14.082+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:14.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:14.082+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:14.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:14.082+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:14.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:14.082+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:15.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:15.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:15.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:15.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:15.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:15.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:14 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:14.579+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:15.108 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:14.683+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:15.108 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:14.683+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:15.109 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:14.683+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:15.109 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:14 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:14.683+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:15.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:14 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:14.935+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:15.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:15.107+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:15.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:15.107+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:15.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:15.107+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:15.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:15.107+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:15.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:15.107+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:16.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:16.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:16.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:16.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:15 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:15.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:15.670+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:07.659895+0000 front 2024-08-16T19:27:07.659924+0000 (oldest deadline 2024-08-16T19:27:33.559979+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:15.670+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.061362+0000 front 2024-08-16T19:27:17.061344+0000 (oldest deadline 2024-08-16T19:27:41.761273+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:15.670+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:26.263076+0000 front 2024-08-16T19:27:26.263119+0000 (oldest deadline 2024-08-16T19:27:52.163273+0000) 2024-08-16T19:28:16.080 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8[55751]: 2024-08-16T19:28:15.670+0000 7fa267f84640 -1 osd.8 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:32.163886+0000 front 2024-08-16T19:27:32.163725+0000 (oldest deadline 2024-08-16T19:27:58.063758+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:16.247 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:15 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:15.976+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:16.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:16 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:16.079+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:16.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:16 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:16.079+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:16.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:16 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:16.079+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:16.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:16 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:16.079+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:16.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:16 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:16.079+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:17.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:17.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:17.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:17.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:17.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:17.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:16 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:16.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:17.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:17 smithi161 podman[75428]: 2024-08-16 19:28:17.035055359 +0000 UTC m=+5.904781309 container died 174b8e397ec47dfd2b5464627cb1b2b837eb76ee78ca3cbfa464a8b7204765f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:17.302 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:17 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:17.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:17.302 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:17 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:17.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:17.302 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:17 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:17.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:17.303 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:17 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:17.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:17.303 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:17 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:17.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:17.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:16 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:16.996+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:17.601 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:17 smithi161 podman[75428]: 2024-08-16 19:28:17.375348743 +0000 UTC m=+6.245074689 container cleanup 174b8e397ec47dfd2b5464627cb1b2b837eb76ee78ca3cbfa464a8b7204765f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:17.602 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:17 smithi161 bash[75428]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8 2024-08-16T19:28:17.888 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:17 smithi161 podman[75447]: 2024-08-16 19:28:17.601510387 +0000 UTC m=+0.565469376 container remove 174b8e397ec47dfd2b5464627cb1b2b837eb76ee78ca3cbfa464a8b7204765f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-16T19:28:18.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:18.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:18.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:18.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:18.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:18.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:17 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:17.597+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:18.291 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:18 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:18.105+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:18.291 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:18 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:18.105+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:18 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:18.105+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:18 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:18.105+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:18 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:18.105+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:17.943406516 +0000 UTC m=+0.017905948 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:18.052305551 +0000 UTC m=+0.126805014 container create e5f532d8ab754ce3e3b88259e8ea10ba7d3d4732c25930d2a5a51e124e79a640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-16T19:28:18.292 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:18.288271596 +0000 UTC m=+0.362771024 container init e5f532d8ab754ce3e3b88259e8ea10ba7d3d4732c25930d2a5a51e124e79a640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21) 2024-08-16T19:28:18.293 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:18.291318903 +0000 UTC m=+0.365818320 container start e5f532d8ab754ce3e3b88259e8ea10ba7d3d4732c25930d2a5a51e124e79a640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8-deactivate, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:18.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:18.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:18.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:18.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:18.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:18.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:18 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:18.042+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:18.638 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:18.38842416 +0000 UTC m=+0.462923589 container attach e5f532d8ab754ce3e3b88259e8ea10ba7d3d4732c25930d2a5a51e124e79a640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:28:18.638 INFO:journalctl@ceph.osd.8.smithi161.stdout:Aug 16 19:28:18 smithi161 podman[75532]: 2024-08-16 19:28:18.474617679 +0000 UTC m=+0.549117104 container died e5f532d8ab754ce3e3b88259e8ea10ba7d3d4732c25930d2a5a51e124e79a640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-16T19:28:19.058 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.8.service' 2024-08-16T19:28:19.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:19.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:19.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:19.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:19.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:19.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:18 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:18.588+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:19.097 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:19.097 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-16T19:28:19.097 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-16T19:28:19.098 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.9 2024-08-16T19:28:19.312 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:19.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:19.312 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:19.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:19.312 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:19.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:19.312 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:19.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:19.312 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:19.128+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:19.422 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:19.422 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:19.422 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.091+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 systemd[1]: Stopping Ceph osd.9 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.421+0000 7f865fc1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.421+0000 7f865fc1d640 -1 osd.9 117 *** Got signal Terminated *** 2024-08-16T19:28:19.423 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:19 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:19.421+0000 7f865fc1d640 -1 osd.9 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:28:19.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:19.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:19.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:19.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:19.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:19.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:19 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:19.568+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:20.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:20 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:20.106+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:20.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:20 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:20.106+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:20.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:20 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:20.106+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:20.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:20 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:20.106+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:20.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:20 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:20.106+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:20.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:20.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:20.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:20.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:20.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:20.498 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:20 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:20.052+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:21.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:21.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:21.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:21.079 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:21.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:21.080 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:20 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:20.606+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:21.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:21 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:21.068+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:21.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:21 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:21.068+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:21.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:21 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:21.068+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:21.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:21 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:21.068+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:21.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:21 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:21.068+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:21.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:21 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:21.056+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:21.980 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:21.980 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:21.981 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:21.981 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:21.981 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:21.981 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:21 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:21.582+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:22.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:22 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:22.108+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:22.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:22 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:22.108+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:22.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:22 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:22.108+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:22.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:22 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:22.108+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:22.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:22 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:22.108+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:22.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:22.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:22.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:22.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:22.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:22.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:22 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:22.049+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:22.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:22.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:22.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:22.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:22.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:22.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:22 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:22.550+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:23.496 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:23.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:23.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:23.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:23.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:23.497 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:23 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:23.073+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:23.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:23.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:23.638 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:23.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:23.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:23.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:23.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:23.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:23.639 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:23.153+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:23.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:23.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:23.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:23.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:23.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:23.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:23 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:23.506+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:24.390 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:24.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:24.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:24.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:24.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:24.608 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:24 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:24.104+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:24.757 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.570440+0000 front 2024-08-16T19:27:12.570495+0000 (oldest deadline 2024-08-16T19:27:34.171245+0000) 2024-08-16T19:28:24.757 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:16.972946+0000 front 2024-08-16T19:27:16.973029+0000 (oldest deadline 2024-08-16T19:27:42.272836+0000) 2024-08-16T19:28:24.757 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:28.074885+0000 front 2024-08-16T19:27:28.074813+0000 (oldest deadline 2024-08-16T19:27:53.374918+0000) 2024-08-16T19:28:24.757 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.475802+0000 front 2024-08-16T19:27:34.475780+0000 (oldest deadline 2024-08-16T19:28:00.375934+0000) 2024-08-16T19:28:24.758 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:43.877022+0000 front 2024-08-16T19:27:43.876808+0000 (oldest deadline 2024-08-16T19:28:07.976891+0000) 2024-08-16T19:28:24.758 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9[66800]: 2024-08-16T19:28:24.120+0000 7f865822b640 -1 osd.9 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:51.478033+0000 front 2024-08-16T19:27:51.478010+0000 (oldest deadline 2024-08-16T19:28:13.177893+0000) 2024-08-16T19:28:24.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:24.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:24.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:24.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:24.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:24.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:24 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:24.467+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:25.078 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:24 smithi062 podman[105210]: 2024-08-16 19:28:24.825915135 +0000 UTC m=+5.529603795 container died 18984ba59aa779d9f72ad45af4f50fedff0bb02c723e5a020d06463721818e70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-16T19:28:25.337 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105210]: 2024-08-16 19:28:25.168868384 +0000 UTC m=+5.872557042 container cleanup 18984ba59aa779d9f72ad45af4f50fedff0bb02c723e5a020d06463721818e70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-16T19:28:25.337 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 bash[105210]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9 2024-08-16T19:28:25.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:25.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:25.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:25.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:25.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:25.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:25 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:25.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:25.746 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105224]: 2024-08-16 19:28:25.336740775 +0000 UTC m=+0.511145734 container remove 18984ba59aa779d9f72ad45af4f50fedff0bb02c723e5a020d06463721818e70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-16T19:28:25.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:25.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:25.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:25.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:25.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:25.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:25.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:25 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:25.448+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:26.031 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105313]: 2024-08-16 19:28:25.6534541 +0000 UTC m=+0.018019724 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:26.032 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105313]: 2024-08-16 19:28:25.754064425 +0000 UTC m=+0.118630040 container create 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:28:26.032 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105313]: 2024-08-16 19:28:25.956605123 +0000 UTC m=+0.321170737 container init 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-16T19:28:26.032 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:25 smithi062 podman[105313]: 2024-08-16 19:28:25.959990736 +0000 UTC m=+0.324556351 container start 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-16T19:28:26.032 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:26 smithi062 podman[105313]: 2024-08-16 19:28:26.031392563 +0000 UTC m=+0.395958183 container attach 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=squid-d863d21, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-08-16T19:28:26.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:26.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:26.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:26.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:26.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:26.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:26 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:26.077+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:26.451 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:26 smithi062 podman[105313]: 2024-08-16 19:28:26.158635046 +0000 UTC m=+0.523200663 container died 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-16T19:28:26.702 INFO:journalctl@ceph.osd.9.smithi062.stdout:Aug 16 19:28:26 smithi062 podman[105337]: 2024-08-16 19:28:26.684641243 +0000 UTC m=+0.515843533 container remove 2d0214c46404a0ac5c2ec154cb22886ebdf47323bea04e1d4819e3e1bf9805e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:28:26.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.9.service' 2024-08-16T19:28:26.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:26.749 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-16T19:28:26.749 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-16T19:28:26.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.10 2024-08-16T19:28:26.777 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:26.777 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:26.777 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:26.777 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:26.778 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:26.778 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:26.778 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:26.407+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:27.068 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:26 smithi155 systemd[1]: Stopping Ceph osd.10 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:27.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.067+0000 7fdd5b7ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:27.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.067+0000 7fdd5b7ad640 -1 osd.10 117 *** Got signal Terminated *** 2024-08-16T19:28:27.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.067+0000 7fdd5b7ad640 -1 osd.10 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:28:27.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:27.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:27.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:27.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:27.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:27.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:27.058+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:27.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:27.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:27.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:27.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:27.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:27.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:27.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:27 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:27.452+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:28.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:28.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:28.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:28.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:28.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:28.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:28 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:28.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:28.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:28.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:28.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:28.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:28.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:28.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:28.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:28 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:28.451+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:29.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:29 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:29.114+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:29.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:29.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:29.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:29.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:29.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:29.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:29.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:29 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:29.462+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:30.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:30.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:30.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:30.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:30.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:30.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:30.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:30 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:30.090+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:30.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:30.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:30.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:30.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:30.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:30.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:30.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:30 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:30.477+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:31.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:31.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:31.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:31.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:31.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:31.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:31.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:31 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:31.094+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:31.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:12.266398+0000 front 2024-08-16T19:27:12.266256+0000 (oldest deadline 2024-08-16T19:27:35.166482+0000) 2024-08-16T19:28:31.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-08-16T19:27:17.467606+0000 front 2024-08-16T19:27:17.467584+0000 (oldest deadline 2024-08-16T19:27:43.367693+0000) 2024-08-16T19:28:31.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:25.669088+0000 front 2024-08-16T19:27:25.669018+0000 (oldest deadline 2024-08-16T19:27:51.569129+0000) 2024-08-16T19:28:31.829 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:35.471189+0000 front 2024-08-16T19:27:35.471212+0000 (oldest deadline 2024-08-16T19:27:57.771205+0000) 2024-08-16T19:28:31.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:44.172846+0000 front 2024-08-16T19:27:44.172906+0000 (oldest deadline 2024-08-16T19:28:08.873084+0000) 2024-08-16T19:28:31.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:48.873904+0000 front 2024-08-16T19:27:48.874011+0000 (oldest deadline 2024-08-16T19:28:13.573859+0000) 2024-08-16T19:28:31.830 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:31 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10[62080]: 2024-08-16T19:28:31.445+0000 7fdd57dc3640 -1 osd.10 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:58.075806+0000 front 2024-08-16T19:27:58.075821+0000 (oldest deadline 2024-08-16T19:28:24.476244+0000) 2024-08-16T19:28:32.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:32.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:32.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:32.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:32.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:32.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:32.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:32 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:32.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:33.001 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:32 smithi155 podman[76374]: 2024-08-16 19:28:32.66670564 +0000 UTC m=+5.709021587 container died 11b160c9751cf5f1b9e0b40b42795c76e2e0f799544e91cb1ca19a50e79deabf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-16T19:28:33.002 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76374]: 2024-08-16 19:28:33.001832734 +0000 UTC m=+6.044148678 container cleanup 11b160c9751cf5f1b9e0b40b42795c76e2e0f799544e91cb1ca19a50e79deabf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.vendor=CentOS) 2024-08-16T19:28:33.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 bash[76374]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10 2024-08-16T19:28:33.329 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76389]: 2024-08-16 19:28:33.194869894 +0000 UTC m=+0.528490809 container remove 11b160c9751cf5f1b9e0b40b42795c76e2e0f799544e91cb1ca19a50e79deabf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-16T19:28:33.377 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:33.377 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:33.378 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:33.378 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:33.378 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:33.378 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:33.378 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:33 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:33.112+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:33.914 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76478]: 2024-08-16 19:28:33.518376053 +0000 UTC m=+0.017431866 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:33.915 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76478]: 2024-08-16 19:28:33.620468672 +0000 UTC m=+0.119524471 container create fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-16T19:28:33.915 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76478]: 2024-08-16 19:28:33.831373663 +0000 UTC m=+0.330429463 container init fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21) 2024-08-16T19:28:33.915 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76478]: 2024-08-16 19:28:33.834617383 +0000 UTC m=+0.333673186 container start fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21) 2024-08-16T19:28:33.915 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:33 smithi155 podman[76478]: 2024-08-16 19:28:33.914508807 +0000 UTC m=+0.413564624 container attach fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-16T19:28:34.293 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:34 smithi155 podman[76478]: 2024-08-16 19:28:34.0249953 +0000 UTC m=+0.524051110 container died fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-08-16T19:28:34.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:34.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:34.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:34.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:34.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:34.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:34.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.064+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:34.546 INFO:journalctl@ceph.osd.10.smithi155.stdout:Aug 16 19:28:34 smithi155 podman[76501]: 2024-08-16 19:28:34.543108192 +0000 UTC m=+0.512379256 container remove fde981982768fb54958717cffd9604b9151071b6c81ed797a7916897df81e3ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-10-deactivate, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-16T19:28:34.566 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.10.service' 2024-08-16T19:28:34.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:34.644 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-16T19:28:34.644 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-16T19:28:34.644 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.11 2024-08-16T19:28:34.944 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 systemd[1]: Stopping Ceph osd.11 for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:35.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.943+0000 7f330dad2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:35.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.943+0000 7f330dad2640 -1 osd.11 117 *** Got signal Terminated *** 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:34 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:34.943+0000 7f330dad2640 -1 osd.11 117 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:35.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:35.390 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:35 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:35.041+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:36.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:36.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:36.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:36.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:36.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:36.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:36.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:36 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:36.061+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:37.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:37.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:37.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:37.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:37.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:37.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:37.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:37 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:37.067+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:38.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:38.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:38.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:38.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:38.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:38 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:38.097+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-08-16T19:28:12.252944+0000 front 2024-08-16T19:28:12.252920+0000 (oldest deadline 2024-08-16T19:28:37.552999+0000) 2024-08-16T19:28:39.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6806 osd.0 since back 2024-08-16T19:27:08.142206+0000 front 2024-08-16T19:27:08.142296+0000 (oldest deadline 2024-08-16T19:27:34.042056+0000) 2024-08-16T19:28:39.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-08-16T19:27:27.846104+0000 front 2024-08-16T19:27:27.846137+0000 (oldest deadline 2024-08-16T19:27:51.946258+0000) 2024-08-16T19:28:39.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6814 osd.3 since back 2024-08-16T19:27:34.847537+0000 front 2024-08-16T19:27:34.847503+0000 (oldest deadline 2024-08-16T19:28:00.747608+0000) 2024-08-16T19:28:39.388 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-08-16T19:27:40.748406+0000 front 2024-08-16T19:27:40.748262+0000 (oldest deadline 2024-08-16T19:28:05.448046+0000) 2024-08-16T19:28:39.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-08-16T19:27:52.449559+0000 front 2024-08-16T19:27:52.449538+0000 (oldest deadline 2024-08-16T19:28:13.549624+0000) 2024-08-16T19:28:39.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.62:6822 osd.6 since back 2024-08-16T19:27:59.450939+0000 front 2024-08-16T19:27:59.450783+0000 (oldest deadline 2024-08-16T19:28:23.550720+0000) 2024-08-16T19:28:39.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-08-16T19:28:05.251854+0000 front 2024-08-16T19:28:05.251911+0000 (oldest deadline 2024-08-16T19:28:29.951886+0000) 2024-08-16T19:28:39.389 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:39 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11[62551]: 2024-08-16T19:28:39.055+0000 7f330a0e8640 -1 osd.11 117 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-08-16T19:28:12.252944+0000 front 2024-08-16T19:28:12.252920+0000 (oldest deadline 2024-08-16T19:28:37.552999+0000) 2024-08-16T19:28:40.826 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:40 smithi161 podman[75648]: 2024-08-16 19:28:40.518173313 +0000 UTC m=+5.675566180 container died 00d2b5dc6bc74e1a33d6ec3e89056ad19c1d24892504599a0395cfc6afccb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:40.826 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:40 smithi161 podman[75648]: 2024-08-16 19:28:40.82608017 +0000 UTC m=+5.983473033 container cleanup 00d2b5dc6bc74e1a33d6ec3e89056ad19c1d24892504599a0395cfc6afccb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-16T19:28:41.138 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:40 smithi161 bash[75648]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11 2024-08-16T19:28:41.138 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75663]: 2024-08-16 19:28:41.019180207 +0000 UTC m=+0.502390151 container remove 00d2b5dc6bc74e1a33d6ec3e89056ad19c1d24892504599a0395cfc6afccb403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-16T19:28:41.755 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.35188562 +0000 UTC m=+0.017251785 image pull ba08250ef864474725fb856db2617a8d8103b32c0e9c824eedd71e03d6315147 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb 2024-08-16T19:28:41.756 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.461755674 +0000 UTC m=+0.127121881 container create d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:28:41.756 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.680725342 +0000 UTC m=+0.346091516 container init d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-16T19:28:41.756 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.683709321 +0000 UTC m=+0.349075492 container start d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-08-16T19:28:41.756 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.755595616 +0000 UTC m=+0.420961803 container attach d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS) 2024-08-16T19:28:42.138 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:41 smithi161 podman[75749]: 2024-08-16 19:28:41.866391829 +0000 UTC m=+0.531758003 container died d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-08-16T19:28:42.367 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.11.service' 2024-08-16T19:28:42.392 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:42 smithi161 podman[75772]: 2024-08-16 19:28:42.341852131 +0000 UTC m=+0.469415949 container remove d1a2938386dc02df2fe8fb22bf5265da995a3dd8e1e5e2aea682990ce47fe5dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-16T19:28:42.393 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:42 smithi161 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.11.service: Deactivated successfully. 2024-08-16T19:28:42.393 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:42 smithi161 systemd[1]: Stopped Ceph osd.11 for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:28:42.393 INFO:journalctl@ceph.osd.11.smithi161.stdout:Aug 16 19:28:42 smithi161 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@osd.11.service: Consumed 1min 41.486s CPU time. 2024-08-16T19:28:42.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:42.406 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-16T19:28:42.406 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-16T19:28:42.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.a 2024-08-16T19:28:42.712 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:42 smithi062 systemd[1]: Stopping Ceph mds.a for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:42.985 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:42 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: 2024-08-16T19:28:42.711+0000 7ff770698640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:42.985 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:42 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a[73679]: 2024-08-16T19:28:42.711+0000 7ff770698640 -1 mds.a *** got signal Terminated *** 2024-08-16T19:28:46.997 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:46 smithi062 podman[105418]: 2024-08-16 19:28:46.718775929 +0000 UTC m=+4.109622839 container died cb9791b7d2cba6a33dba6c96933be62b874b6a74fb696c2274dc060e512fe81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-16T19:28:47.438 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 podman[105418]: 2024-08-16 19:28:47.23641037 +0000 UTC m=+4.627257321 container cleanup cb9791b7d2cba6a33dba6c96933be62b874b6a74fb696c2274dc060e512fe81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-16T19:28:47.438 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 bash[105418]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a 2024-08-16T19:28:47.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.a.service' 2024-08-16T19:28:47.703 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 podman[105442]: 2024-08-16 19:28:47.437646637 +0000 UTC m=+0.712931113 container remove cb9791b7d2cba6a33dba6c96933be62b874b6a74fb696c2274dc060e512fe81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-a, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:28:47.703 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.a.service: Deactivated successfully. 2024-08-16T19:28:47.703 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 systemd[1]: Stopped Ceph mds.a for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:28:47.703 INFO:journalctl@ceph.mds.a.smithi062.stdout:Aug 16 19:28:47 smithi062 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.a.service: Consumed 7.435s CPU time. 2024-08-16T19:28:47.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:47.717 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-16T19:28:47.717 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-16T19:28:47.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.d 2024-08-16T19:28:47.997 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:47 smithi062 systemd[1]: Stopping Ceph mds.d for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:48.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d[72485]: 2024-08-16T19:28:48.082+0000 7fccb71b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:48.497 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:48 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d[72485]: 2024-08-16T19:28:48.082+0000 7fccb71b4640 -1 mds.d *** got signal Terminated *** 2024-08-16T19:28:52.441 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:52 smithi062 podman[105551]: 2024-08-16 19:28:52.088867517 +0000 UTC m=+4.146716350 container died 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d, RELEASE=squid-d863d21, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:52.717 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:52 smithi062 podman[105551]: 2024-08-16 19:28:52.50730353 +0000 UTC m=+4.565152348 container cleanup 0a98a8064d1ac4d3ca00966dd5b61697dd4b246b7e766c223006e3abd9e5f07b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21) 2024-08-16T19:28:52.717 INFO:journalctl@ceph.mds.d.smithi062.stdout:Aug 16 19:28:52 smithi062 bash[105551]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-d 2024-08-16T19:28:52.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.d.service' 2024-08-16T19:28:52.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:52.909 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-16T19:28:52.909 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-16T19:28:52.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.g 2024-08-16T19:28:53.228 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:52 smithi062 systemd[1]: Stopping Ceph mds.g for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:53.496 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: 2024-08-16T19:28:53.227+0000 7fa241edd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:53.497 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:53 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g[73280]: 2024-08-16T19:28:53.227+0000 7fa241edd640 -1 mds.g *** got signal Terminated *** 2024-08-16T19:28:54.979 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:54 smithi062 podman[105671]: 2024-08-16 19:28:54.721049546 +0000 UTC m=+1.591271079 container died 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-08-16T19:28:55.246 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:55 smithi062 podman[105671]: 2024-08-16 19:28:55.119884624 +0000 UTC m=+1.990106156 container cleanup 5db14e43d4f314f6ccc30b02d031e1f5381452234806d50628726bca72dd581d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-16T19:28:55.247 INFO:journalctl@ceph.mds.g.smithi062.stdout:Aug 16 19:28:55 smithi062 bash[105671]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-g 2024-08-16T19:28:55.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.g.service' 2024-08-16T19:28:55.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:55.541 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-16T19:28:55.541 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-16T19:28:55.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.j 2024-08-16T19:28:55.996 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:55 smithi062 systemd[1]: Stopping Ceph mds.j for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:55.997 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j[72072]: 2024-08-16T19:28:55.857+0000 7fea96259640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:55.997 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:55 smithi062 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j[72072]: 2024-08-16T19:28:55.857+0000 7fea96259640 -1 mds.j *** got signal Terminated *** 2024-08-16T19:28:57.496 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:57 smithi062 podman[105791]: 2024-08-16 19:28:57.130292561 +0000 UTC m=+1.365952122 container died 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:28:57.936 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:57 smithi062 podman[105791]: 2024-08-16 19:28:57.776274503 +0000 UTC m=+2.011934052 container cleanup 72c5dbc0bdafe2aff884bdfdeb1ed24f1b4d72600af0bc462962358a56c1b482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-16T19:28:57.936 INFO:journalctl@ceph.mds.j.smithi062.stdout:Aug 16 19:28:57 smithi062 bash[105791]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-j 2024-08-16T19:28:58.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.j.service' 2024-08-16T19:28:58.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:28:58.186 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-16T19:28:58.186 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-16T19:28:58.186 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.b 2024-08-16T19:28:58.494 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:28:58 smithi155 systemd[1]: Stopping Ceph mds.b for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:28:58.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:28:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:28:58.492+0000 7fe90a408640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:28:58.829 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:28:58 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b[67710]: 2024-08-16T19:28:58.492+0000 7fe90a408640 -1 mds.b *** got signal Terminated *** 2024-08-16T19:29:02.811 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:29:02 smithi155 podman[76581]: 2024-08-16 19:29:02.502037433 +0000 UTC m=+4.112248646 container died 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-16T19:29:03.079 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:29:02 smithi155 podman[76581]: 2024-08-16 19:29:02.868332217 +0000 UTC m=+4.478543422 container cleanup 5a154aa5c52932c2bc11f4295ec207b0e298538e0194056371fe027f676a1e00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d863d21) 2024-08-16T19:29:03.082 INFO:journalctl@ceph.mds.b.smithi155.stdout:Aug 16 19:29:02 smithi155 bash[76581]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-b 2024-08-16T19:29:03.303 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.b.service' 2024-08-16T19:29:03.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:03.338 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-16T19:29:03.339 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-16T19:29:03.339 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.e 2024-08-16T19:29:03.656 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:03 smithi155 systemd[1]: Stopping Ceph mds.e for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:04.079 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: 2024-08-16T19:29:03.655+0000 7f1fc5784640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:04.079 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:03 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e[67106]: 2024-08-16T19:29:03.655+0000 7f1fc5784640 -1 mds.e *** got signal Terminated *** 2024-08-16T19:29:07.079 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:06 smithi155 podman[76712]: 2024-08-16 19:29:06.719569461 +0000 UTC m=+3.154566568 container died fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d863d21, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-16T19:29:07.457 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 podman[76712]: 2024-08-16 19:29:07.264090284 +0000 UTC m=+3.699087390 container cleanup fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-16T19:29:07.457 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 bash[76712]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e 2024-08-16T19:29:07.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.e.service' 2024-08-16T19:29:07.708 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 podman[76725]: 2024-08-16 19:29:07.457050732 +0000 UTC m=+0.730124033 container remove fd8951e1e8d8ec12019c361a8b59038544c25f6519879065b8c82d5818131f21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-16T19:29:07.708 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.e.service: Deactivated successfully. 2024-08-16T19:29:07.708 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 systemd[1]: Stopped Ceph mds.e for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:29:07.708 INFO:journalctl@ceph.mds.e.smithi155.stdout:Aug 16 19:29:07 smithi155 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.e.service: Consumed 8.122s CPU time. 2024-08-16T19:29:07.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:07.719 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-16T19:29:07.720 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-16T19:29:07.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.h 2024-08-16T19:29:08.051 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:07 smithi155 systemd[1]: Stopping Ceph mds.h for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:08.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: 2024-08-16T19:29:08.050+0000 7fde28fd9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:08.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:08 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h[67408]: 2024-08-16T19:29:08.050+0000 7fde28fd9640 -1 mds.h *** got signal Terminated *** 2024-08-16T19:29:11.027 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:10 smithi155 podman[76834]: 2024-08-16 19:29:10.71912015 +0000 UTC m=+2.780044503 container died cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=) 2024-08-16T19:29:11.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:11 smithi155 podman[76834]: 2024-08-16 19:29:11.109635645 +0000 UTC m=+3.170559994 container cleanup cdc114c91707ee7b9e737449d48c97f4f342e4d984c16580aa1a73dfa37056b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-16T19:29:11.329 INFO:journalctl@ceph.mds.h.smithi155.stdout:Aug 16 19:29:11 smithi155 bash[76834]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-h 2024-08-16T19:29:11.500 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.h.service' 2024-08-16T19:29:11.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:11.530 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-16T19:29:11.531 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-16T19:29:11.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.k 2024-08-16T19:29:11.889 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:11 smithi155 systemd[1]: Stopping Ceph mds.k for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:12.329 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: 2024-08-16T19:29:11.888+0000 7fd8ae7f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:12.329 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:11 smithi155 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k[66786]: 2024-08-16T19:29:11.888+0000 7fd8ae7f1640 -1 mds.k *** got signal Terminated *** 2024-08-16T19:29:15.006 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:14 smithi155 podman[76955]: 2024-08-16 19:29:14.738489239 +0000 UTC m=+2.983129085 container died 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k, RELEASE=squid-d863d21, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:29:15.329 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:15 smithi155 podman[76955]: 2024-08-16 19:29:15.088691836 +0000 UTC m=+3.333331685 container cleanup 3911eb8823cd1b831265e96a8147f31f8b60da9551cb78c67c8bc99b8836d510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-16T19:29:15.329 INFO:journalctl@ceph.mds.k.smithi155.stdout:Aug 16 19:29:15 smithi155 bash[76955]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-k 2024-08-16T19:29:15.503 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.k.service' 2024-08-16T19:29:15.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:15.535 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-16T19:29:15.535 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-16T19:29:15.535 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.c 2024-08-16T19:29:15.888 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:15 smithi161 systemd[1]: Stopping Ceph mds.c for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:16.388 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: 2024-08-16T19:29:15.918+0000 7f626331e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:16.388 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:15 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c[67380]: 2024-08-16T19:29:15.918+0000 7f626331e640 -1 mds.c *** got signal Terminated *** 2024-08-16T19:29:19.011 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:18 smithi161 podman[75852]: 2024-08-16 19:29:18.731053783 +0000 UTC m=+3.001878674 container died 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d863d21, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-16T19:29:19.352 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:19 smithi161 podman[75852]: 2024-08-16 19:29:19.101174569 +0000 UTC m=+3.371999466 container cleanup 45dc381d510335d02c557888f65fc13a5a11cd792613734c471d44ddc16f46ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=squid-d863d21, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:29:19.352 INFO:journalctl@ceph.mds.c.smithi161.stdout:Aug 16 19:29:19 smithi161 bash[75852]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-c 2024-08-16T19:29:19.531 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.c.service' 2024-08-16T19:29:19.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:19.565 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-16T19:29:19.566 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-16T19:29:19.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.f 2024-08-16T19:29:19.880 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:19 smithi161 systemd[1]: Stopping Ceph mds.f for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:20.138 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: 2024-08-16T19:29:19.879+0000 7fa953d29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:20.138 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:19 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f[67080]: 2024-08-16T19:29:19.879+0000 7fa953d29640 -1 mds.f *** got signal Terminated *** 2024-08-16T19:29:23.005 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:22 smithi161 podman[75986]: 2024-08-16 19:29:22.736370023 +0000 UTC m=+2.953958651 container died 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-d863d21, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-16T19:29:23.314 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 podman[75986]: 2024-08-16 19:29:23.079583173 +0000 UTC m=+3.297171792 container cleanup 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d863d21) 2024-08-16T19:29:23.314 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 bash[75986]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f 2024-08-16T19:29:23.550 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.f.service' 2024-08-16T19:29:23.570 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 podman[75998]: 2024-08-16 19:29:23.314089688 +0000 UTC m=+0.575085499 container remove 98f2754eb4de432a3d4f5bb4a281b4ae795d59414cd71ac15ee16358fc24c337 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-d863d21, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:29:23.570 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.f.service: Deactivated successfully. 2024-08-16T19:29:23.570 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 systemd[1]: Stopped Ceph mds.f for bd001ebc-5c01-11ef-bccf-c7b262605968. 2024-08-16T19:29:23.570 INFO:journalctl@ceph.mds.f.smithi161.stdout:Aug 16 19:29:23 smithi161 systemd[1]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.f.service: Consumed 10.479s CPU time. 2024-08-16T19:29:23.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:23.582 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-16T19:29:23.582 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-16T19:29:23.583 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.i 2024-08-16T19:29:23.888 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:23 smithi161 systemd[1]: Stopping Ceph mds.i for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:24.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: 2024-08-16T19:29:23.906+0000 7f0630789640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:24.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:23 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i[67693]: 2024-08-16T19:29:23.906+0000 7f0630789640 -1 mds.i *** got signal Terminated *** 2024-08-16T19:29:27.099 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:26 smithi161 podman[76106]: 2024-08-16 19:29:26.726127094 +0000 UTC m=+2.925665542 container died e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:29:27.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:27 smithi161 podman[76106]: 2024-08-16 19:29:27.099695956 +0000 UTC m=+3.299234403 container cleanup e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-08-16T19:29:27.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:27 smithi161 bash[76106]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i 2024-08-16T19:29:27.388 INFO:journalctl@ceph.mds.i.smithi161.stdout:Aug 16 19:29:27 smithi161 podman[76118]: 2024-08-16 19:29:27.32572586 +0000 UTC m=+0.597870026 container remove e854ba103aeb8267fa6a3ec6d40175c808d64a1abebb0f4c9410fef569ff77e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-16T19:29:27.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.i.service' 2024-08-16T19:29:27.518 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:27.518 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-16T19:29:27.518 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-16T19:29:27.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.l 2024-08-16T19:29:27.820 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:27 smithi161 systemd[1]: Stopping Ceph mds.l for bd001ebc-5c01-11ef-bccf-c7b262605968... 2024-08-16T19:29:28.138 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: 2024-08-16T19:29:27.818+0000 7f5ec9cdf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-16T19:29:28.138 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:27 smithi161 ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l[66762]: 2024-08-16T19:29:27.818+0000 7f5ec9cdf640 -1 mds.l *** got signal Terminated *** 2024-08-16T19:29:31.003 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:30 smithi161 podman[76227]: 2024-08-16 19:29:30.731280725 +0000 UTC m=+2.996911131 container died 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-d863d21, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-16T19:29:31.337 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:31 smithi161 podman[76227]: 2024-08-16 19:29:31.094743 +0000 UTC m=+3.360373409 container cleanup 0dc5272ab4139ce642bac4291300110b5aca40f1806d7366b96e6632616d081e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:51a0b9579fb4a4554a153b3dbfee2d70518b2d8efa2fd27af3b96767db1865eb, name=ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d863d21, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-08-16T19:29:31.337 INFO:journalctl@ceph.mds.l.smithi161.stdout:Aug 16 19:29:31 smithi161 bash[76227]: ceph-bd001ebc-5c01-11ef-bccf-c7b262605968-mds-l 2024-08-16T19:29:31.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd001ebc-5c01-11ef-bccf-c7b262605968@mds.l.service' 2024-08-16T19:29:31.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-16T19:29:31.506 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-16T19:29:31.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-16T19:29:31.726 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:29:34.394 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-16T19:29:34.598 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:29:37.085 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-16T19:29:37.291 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:29:39.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:29:39.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:29:39.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-16T19:29:39.749 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-16T19:29:39.751 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi062/crash 2024-08-16T19:29:39.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash -- . 2024-08-16T19:29:39.790 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-16T19:29:39.790 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-08-16T19:29:39.792 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi155/crash 2024-08-16T19:29:39.793 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash -- . 2024-08-16T19:29:39.833 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-16T19:29:39.833 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-08-16T19:29:39.835 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi161/crash 2024-08-16T19:29:39.836 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash -- . 2024-08-16T19:29:39.865 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-16T19:29:39.865 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-08-16T19:29:39.866 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-16T19:29:39.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-08-16T19:29:39.903 INFO:tasks.cephadm:Compressing logs... 2024-08-16T19:29:39.904 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-16T19:29:39.946 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-16T19:29:39.948 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-16T19:29:39.972 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-16T19:29:39.981 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log: /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.z.log: 87.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr: 90.5%gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr: -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-16T19:29:39.982 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-16T19:29:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log 2024-08-16T19:29:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2024-08-16T19:29:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-16T19:29:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-16T19:29:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log: /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.j.log: /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr: 87.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-16T19:29:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-16T19:29:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.6.log: /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log: 84.5% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-16T19:29:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-16T19:29:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-16T19:29:39.985 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log: /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-16T19:29:39.987 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-16T19:29:39.988 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-16T19:29:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.11.log: 92.3% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-16T19:29:39.989 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-16T19:29:39.992 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-16T19:29:39.995 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-16T19:29:39.996 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-16T19:29:39.999 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-16T19:29:40.006 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-08-16T19:29:40.007 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6%/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-16T19:29:40.014 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-16T19:29:40.028 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-16T19:29:40.029 INFO:teuthology.orchestra.run.smithi161.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-16T19:29:40.031 INFO:teuthology.orchestra.run.smithi062.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-16T19:29:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-08-16T19:29:40.056 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 82.2% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-16T19:29:40.060 INFO:teuthology.orchestra.run.smithi155.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-16T19:29:40.115 INFO:teuthology.orchestra.run.smithi155.stderr: 87.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-16T19:29:40.124 INFO:teuthology.orchestra.run.smithi155.stderr: 90.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-16T19:29:40.128 INFO:teuthology.orchestra.run.smithi155.stderr: 91.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-16T19:29:40.184 INFO:teuthology.orchestra.run.smithi155.stderr: 92.2% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-16T19:29:40.418 INFO:teuthology.orchestra.run.smithi062.stderr: 90.3% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-16T19:29:41.216 INFO:teuthology.orchestra.run.smithi161.stderr: 92.9% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-16T19:29:41.416 INFO:teuthology.orchestra.run.smithi155.stderr: 91.8% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-16T19:29:41.483 INFO:teuthology.orchestra.run.smithi062.stderr: 90.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-16T19:29:41.553 INFO:teuthology.orchestra.run.smithi062.stderr: 95.0% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-16T19:29:42.214 INFO:teuthology.orchestra.run.smithi062.stderr: 95.2% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-16T19:29:43.533 INFO:teuthology.orchestra.run.smithi161.stderr: 95.6% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-16T19:29:46.582 INFO:teuthology.orchestra.run.smithi155.stderr: 95.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-16T19:29:46.629 INFO:teuthology.orchestra.run.smithi155.stderr: 95.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-16T19:29:58.613 INFO:teuthology.orchestra.run.smithi062.stderr: 91.9% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-16T19:30:02.269 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-16T19:30:02.934 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-16T19:30:03.224 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-16T19:30:04.629 INFO:teuthology.orchestra.run.smithi062.stderr: 91.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-16T19:30:04.631 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-08-16T19:30:04.631 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m24.671s 2024-08-16T19:30:04.631 INFO:teuthology.orchestra.run.smithi062.stderr:user 1m31.766s 2024-08-16T19:30:04.631 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m2.696s 2024-08-16T19:30:05.649 INFO:teuthology.orchestra.run.smithi161.stderr: 91.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-16T19:30:07.906 INFO:teuthology.orchestra.run.smithi155.stderr: 91.2% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-16T19:30:09.056 INFO:teuthology.orchestra.run.smithi155.stderr: 91.5% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-16T19:30:09.934 INFO:teuthology.orchestra.run.smithi161.stderr: 91.1% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-16T19:30:11.803 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-16T19:30:12.756 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-16T19:30:13.382 INFO:teuthology.orchestra.run.smithi161.stderr: 91.1% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-16T19:30:16.419 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-16T19:30:27.669 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-16T19:30:28.049 INFO:teuthology.orchestra.run.smithi155.stderr: 93.8% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-16T19:30:44.547 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-16T19:30:44.549 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-08-16T19:30:44.549 INFO:teuthology.orchestra.run.smithi161.stderr:real 1m4.586s 2024-08-16T19:30:44.549 INFO:teuthology.orchestra.run.smithi161.stderr:user 4m25.782s 2024-08-16T19:30:44.549 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m7.581s 2024-08-16T19:31:27.313 INFO:teuthology.orchestra.run.smithi155.stderr: 93.5% -- replaced with /var/log/ceph/bd001ebc-5c01-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-16T19:31:27.315 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-08-16T19:31:27.316 INFO:teuthology.orchestra.run.smithi155.stderr:real 1m47.353s 2024-08-16T19:31:27.316 INFO:teuthology.orchestra.run.smithi155.stderr:user 4m8.266s 2024-08-16T19:31:27.316 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m7.737s 2024-08-16T19:31:27.317 INFO:tasks.cephadm:Archiving logs... 2024-08-16T19:31:27.317 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi062/log 2024-08-16T19:31:27.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-16T19:31:41.606 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi155/log 2024-08-16T19:31:41.607 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-16T19:32:15.920 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi161/log 2024-08-16T19:32:15.921 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-16T19:32:48.951 INFO:tasks.cephadm:Removing cluster... 2024-08-16T19:32:48.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force 2024-08-16T19:32:49.170 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:32:49.575 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force 2024-08-16T19:32:49.784 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:32:50.334 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd001ebc-5c01-11ef-bccf-c7b262605968 --force 2024-08-16T19:32:50.540 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: bd001ebc-5c01-11ef-bccf-c7b262605968 2024-08-16T19:32:51.126 INFO:tasks.cephadm:Removing cephadm ... 2024-08-16T19:32:51.127 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-16T19:32:51.147 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-16T19:32:51.165 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-16T19:32:51.180 INFO:tasks.cephadm:Teardown complete 2024-08-16T19:32:51.180 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-16T19:32:51.191 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-16T19:32:51.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-16T19:32:51.195 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-16T19:32:51.207 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-16T19:32:51.327 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-16T19:32:51.337 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-16T19:32:51.338 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T19:32:51.340 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T19:32:51.343 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-16T19:32:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-08-16T19:32:51.356 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-08-16T19:32:51.357 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-08-16T19:32:51.360 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T19:32:51.360 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-08-16T19:32:51.360 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +545us[ +544us] +/- 47ms 2024-08-16T19:32:51.360 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -488us[ -490us] +/- 66ms 2024-08-16T19:32:51.360 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -215us[ -216us] +/- 63ms 2024-08-16T19:32:51.361 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T19:32:51.362 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T19:32:51.362 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-08-16T19:32:51.362 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +538us[ +535us] +/- 47ms 2024-08-16T19:32:51.362 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -498us[ -502us] +/- 66ms 2024-08-16T19:32:51.363 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -236us[ -236us] +/- 64ms 2024-08-16T19:32:51.363 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T19:32:51.374 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-16T19:32:51.375 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-08-16T19:32:51.375 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +528us[ +517us] +/- 47ms 2024-08-16T19:32:51.375 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -505us[ -516us] +/- 66ms 2024-08-16T19:32:51.375 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -238us[ -249us] +/- 63ms 2024-08-16T19:32:51.375 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-16T19:32:51.376 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-16T19:32:51.386 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-16T19:32:51.387 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-16T19:32:51.398 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-16T19:32:51.490 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-16T19:32:51.591 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-16T19:32:51.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T19:32:51.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:51.688 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T19:32:51.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:51.717 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-16T19:32:51.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:51.745 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-16T19:32:51.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-16T19:32:51.767 INFO:teuthology.task.internal:Duration was 2368.894652 seconds 2024-08-16T19:32:51.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-16T19:32:51.782 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-16T19:32:51.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-16T19:32:51.785 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-16T19:32:51.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-16T19:32:51.825 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T19:32:51.826 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T19:32:51.828 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-16T19:32:52.126 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-16T19:32:52.127 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-08-16T19:32:52.127 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-16T19:32:52.149 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-08-16T19:32:52.149 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-16T19:32:52.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-08-16T19:32:52.193 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-16T19:32:52.213 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-16T19:32:52.214 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-16T19:32:52.217 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-16T19:32:52.235 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-16T19:32:52.279 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-16T19:32:52.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-16T19:32:52.286 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-16T19:32:52.304 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-16T19:32:53.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-16T19:32:53.142 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-16T19:32:53.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-16T19:32:53.168 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-16T19:32:53.193 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-16T19:32:53.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-16T19:32:53.234 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.237 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.258 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-08-16T19:32:53.262 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-08-16T19:32:53.281 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-08-16T19:32:53.306 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:53.328 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:53.342 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-16T19:32:53.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-16T19:32:53.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-16T19:32:53.380 INFO:teuthology.task.internal:Transferring archived files... 2024-08-16T19:32:53.381 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi062 2024-08-16T19:32:53.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-16T19:32:53.440 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi155 2024-08-16T19:32:53.441 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-16T19:32:53.478 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-14_21:24:17-fs-squid-distro-default-smithi/7855996/remote/smithi161 2024-08-16T19:32:53.479 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-16T19:32:53.514 INFO:teuthology.task.internal:Removing archive directory... 2024-08-16T19:32:53.514 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-16T19:32:53.517 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-16T19:32:53.519 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-16T19:32:53.563 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-16T19:32:53.579 INFO:teuthology.task.internal:Not uploading archives. 2024-08-16T19:32:53.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-16T19:32:53.591 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-16T19:32:53.591 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-16T19:32:53.593 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-16T19:32:53.596 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-16T19:32:53.606 INFO:teuthology.orchestra.run.smithi062.stdout: 265527 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 16 19:32 /home/ubuntu/cephtest 2024-08-16T19:32:53.608 INFO:teuthology.orchestra.run.smithi155.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 16 19:32 /home/ubuntu/cephtest 2024-08-16T19:32:53.617 INFO:teuthology.orchestra.run.smithi161.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 16 19:32 /home/ubuntu/cephtest 2024-08-16T19:32:53.618 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-16T19:32:53.629 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-16T19:32:53.658 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2368.894651889801 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi062 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=998c2f980771f4723d937d5cd7891f92dd116699 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c0ddac9668af42008874718194cf6cb9 status: fail success: false 2024-08-16T19:32:53.658 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-16T19:32:53.752 INFO:teuthology.run:FAIL